Patents by Inventor Niklas Enbom

Niklas Enbom has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11727577
    Abstract: Implementations described herein relate to methods, systems, and computer-readable media to render a foreground video. In some implementations, a method includes receiving a plurality of video frames that include depth data and color data. The method further includes downsampling the frames of the video. The method further includes, for each frame, generating an initial segmentation mask that categorizes each pixel of the frame as foreground pixel or background pixel. The method further includes determining a trimap that classifies each pixel of the frame as known background, known foreground, or unknown. The method further includes, for each pixel that is classified as unknown, calculating and storing a weight in a weight map. The method further includes performing fine segmentation to obtain a binary mask for each frame. The method further includes upsampling the plurality of frames based on the binary mask for each frame to obtain a foreground video.
    Type: Grant
    Filed: November 9, 2021
    Date of Patent: August 15, 2023
    Assignee: Google LLC
    Inventors: Guangyu Zhou, Qiang Chen, Niklas Enbom
  • Publication number: 20220067946
    Abstract: Implementations described herein relate to methods, systems, and computer-readable media to render a foreground video. In some implementations, a method includes receiving a plurality of video frames that include depth data and color data. The method further includes downsampling the frames of the video. The method further includes, for each frame, generating an initial segmentation mask that categorizes each pixel of the frame as foreground pixel or background pixel. The method further includes determining a trimap that classifies each pixel of the frame as known background, known foreground, or unknown. The method further includes, for each pixel that is classified as unknown, calculating and storing a weight in a weight map. The method further includes performing fine segmentation to obtain a binary mask for each frame. The method further includes upsampling the plurality of frames based on the binary mask for each frame to obtain a foreground video.
    Type: Application
    Filed: November 9, 2021
    Publication date: March 3, 2022
    Applicant: Google LLC
    Inventors: Guangyu Zhou, Qiang Chen, Niklas Enbom
  • Patent number: 11195283
    Abstract: Implementations described herein relate to methods, systems, and computer-readable media to render a foreground video. In some implementations, a method includes receiving a plurality of video frames that include depth data and color data. The method further includes downsampling the frames of the video. The method further includes, for each frame, generating an initial segmentation mask that categorizes each pixel of the frame as foreground pixel or background pixel. The method further includes determining a trimap that classifies each pixel of the frame as known background, known foreground, or unknown. The method further includes, for each pixel that is classified as unknown, calculating and storing a weight in a weight map. The method further includes performing fine segmentation to obtain a binary mask for each frame. The method further includes upsampling the plurality of frames based on the binary mask for each frame to obtain a foreground video.
    Type: Grant
    Filed: July 15, 2019
    Date of Patent: December 7, 2021
    Assignee: Google LLC
    Inventors: Guangyu Zhou, Qiang Chen, Niklas Enbom
  • Publication number: 20210019892
    Abstract: Implementations described herein relate to methods, systems, and computer-readable media to render a foreground video. In some implementations, a method includes receiving a plurality of video frames that include depth data and color data. The method further includes downsampling the frames of the video. The method further includes, for each frame, generating an initial segmentation mask that categorizes each pixel of the frame as foreground pixel or background pixel. The method further includes determining a trimap that classifies each pixel of the frame as known background, known foreground, or unknown. The method further includes, for each pixel that is classified as unknown, calculating and storing a weight in a weight map. The method further includes performing fine segmentation to obtain a binary mask for each frame. The method further includes upsampling the plurality of frames based on the binary mask for each frame to obtain a foreground video.
    Type: Application
    Filed: July 15, 2019
    Publication date: January 21, 2021
    Applicant: Google LLC
    Inventors: Guangyu Zhou, Qiang Chen, Niklas Enbom
  • Patent number: 9763002
    Abstract: Methods and systems are provided for reusing audio streams in an audio mixing environment such that processing resources (e.g., CPU cycles) are not wasted on encoding the same audio streams multiple times (e.g., more than once). Instead, processing resources are conserved without comprising audio quality. The methods described herein are particularly useful when a participant transitions from using a shared encoder to using the participant's own encoder, and when the participant transitions from using the participant's own encoder to using a shared encoder.
    Type: Grant
    Filed: May 27, 2015
    Date of Patent: September 12, 2017
    Assignee: GOOGLE INC.
    Inventors: Lars Henrik Ellner, Turaj Zakizadeh Shabestary, Niklas Enbom
  • Patent number: 9602918
    Abstract: Methods and systems are provided for reusing audio streams in an audio mixing environment such that processing resources are not wasted on encoding the same audio streams multiple times. Instead, processing resources are conserved without comprising audio quality. The methods described herein are particularly useful when a participant transitions from using a shared encoder to using the participant's own encoder, and when the participant transitions from using the participant's own encoder to using a shared encoder.
    Type: Grant
    Filed: February 28, 2013
    Date of Patent: March 21, 2017
    Assignee: GOOGLE INC.
    Inventors: Lars Henrik Ellner, Turaj Zakizadeh Shabestary, Niklas Enbom
  • Publication number: 20150201274
    Abstract: Methods and systems are provided for reusing audio streams in an audio mixing environment such that processing resources (e.g., CPU cycles) are not wasted on encoding the same audio streams multiple times (e.g., more than once). Instead, processing resources are conserved without comprising audio quality. The methods described herein are particularly useful when a participant transitions from using a shared encoder to using the participant's own encoder, and when the participant transitions from using the participant's own encoder to using a shared encoder.
    Type: Application
    Filed: February 28, 2013
    Publication date: July 16, 2015
    Applicant: Google Inc.
    Inventors: Lars Henrik ELLNER, Turaj Zakizadeh SHABESTARY, Niklas ENBOM
  • Patent number: 8818799
    Abstract: A method and an apparatus for indicating presence of a transient noise in a call are provided. The method comprises the steps of determining activity at an endpoint of the call by monitoring presence of a signal input from the endpoint into the call and monitoring presence of a potential source of transient noise at the endpoint. Further, based on the activity determination and the monitoring of the presence of a potential source of transient noise, a signal representative of the presence of a transient noise in the call is sent. The present invention is advantageous in that it enables improvement of the quality of the call.
    Type: Grant
    Filed: July 8, 2011
    Date of Patent: August 26, 2014
    Assignee: Google Inc.
    Inventors: Niklas Enbom, Roar Hagen, Jan Skoglund
  • Publication number: 20130201272
    Abstract: A control system for varying an audio level in a communication system, the control system comprising a receiving unit for receiving an audio signal and a video signal, a determining unit for determining a number of individuals speaking determined by performing recognition on either the audio signal or the video signal; and a gain adjustment unit for adjusting a gain of the audio signal based on said number of determined individuals that are speaking.
    Type: Application
    Filed: February 7, 2012
    Publication date: August 8, 2013
    Inventor: Niklas ENBOM
  • Patent number: 8331585
    Abstract: The invention relates to a method and an apparatus for mixing audio signals. The invention is based on the idea to base selection of what audio channels to mix on the dynamic behavior of the audio channels in terms of whether they are active or not, rather than basing the selection on quantative measures of the audio channels. According to the invention, a set of audio channels are arranged in a mixing stack and a mixed audio signal is produced from the topmost channels above a predetermined threshold level in the stack. Whenever a channel becomes active, it is placed at the top of the stack.
    Type: Grant
    Filed: May 10, 2007
    Date of Patent: December 11, 2012
    Assignee: Google Inc.
    Inventors: Roar Hagen, Niklas Enbom
  • Publication number: 20120014514
    Abstract: A method and an apparatus for indicating presence of a transient noise in a call are provided. The method comprises the steps of determining activity at an endpoint of the call by monitoring presence of a signal input from the endpoint into the call and monitoring presence of a potential source of transient noise at the endpoint. Further, based on the activity determination and the monitoring of the presence of a potential source of transient noise, a signal representative of the presence of a transient noise in the call is sent. The present invention is advantageous in that it enables improvement of the quality of the call.
    Type: Application
    Filed: July 8, 2011
    Publication date: January 19, 2012
    Applicant: Google Inc.
    Inventors: Niklas Enbom, Roar Hagen, Jan Skoglund
  • Patent number: 7453897
    Abstract: According to the invention, a method for processing an audio media stream that originates from a packet communication network is disclosed. In one step, packets are received as they arrive from the packet communication network. The packets are part of the audio media stream. A playout buffer in a media playout device is monitored. It is determined that the playout buffer is filled below a threshold. A portion of the audio media stream is retrieved when the playout buffer is filled below the threshold. The portion is stored in the playout buffer of the media playout device.
    Type: Grant
    Filed: September 30, 2002
    Date of Patent: November 18, 2008
    Assignees: Global IP Solutions, Inc., Global IP Solutions (GIPS) AB
    Inventors: Niklas Enbom, Fredrik Galschiodt
  • Publication number: 20070274540
    Abstract: The invention relates to a method and an apparatus for mixing audio signals. The invention is based on the idea to base selection of what audio channels to mix on the dynamic behaviour of the audio channels in terms of whether they are active or not, rather than basing the selection on quantative measures of the audio channels. According to the invention, a set of audio channels are arranged in a mixing stack and a mixed audio signal is produced from the topmost channels above a predetermined threshold level in the stack. Whenever a channel becomes active, it is placed at the top of the stack.
    Type: Application
    Filed: May 10, 2007
    Publication date: November 29, 2007
    Applicants: GLOBAL IP SOLUTIONS INC, GLOBAL IP SOLUTIONS (GIPS) AB
    Inventors: Roar Hagen, Niklas Enbom
  • Publication number: 20030091160
    Abstract: According to the invention, a method for processing an audio media stream that originates from a packet communication network is disclosed. In one step, packets are received as they arrive from the packet communication network. The packets are part of the audio media stream. A playout buffer in a media playout device is monitored. It is determined that the playout buffer is filled below a threshold. A portion of the audio media stream is retrieved when the playout buffer is filled below the threshold. The portion is stored in the playout buffer of the media playout device.
    Type: Application
    Filed: September 30, 2002
    Publication date: May 15, 2003
    Applicant: Global IP Sound AB
    Inventors: Niklas Enbom, Fredrik Galschiodt