GoPro Patent Applications

GoPro patent applications that are pending before the United States Patent and Trademark Office (USPTO).

  • Publication number: 20240129636
    Abstract: Visual content that includes spatial portions is obtained. A determination is made that one of the spatial portions includes a face. Based on the determination, encoding quality parameters for the one of the spatial portions is identified. The encoding quality parameters are obtained by combining a first distortion model related to the obtaining the visual content with a second model that emphasizes the one of the spatial portions The visual content is encoded. The encoding quality parameters are stored, in association with but separate from, the one of the spatial portions. After decoding, the one of the spatial portions are rendered based on the encoding quality parameters. The encoding quality parameters are obtained by combining a first distortion model related to the obtaining the visual content with a second model that emphasizes the one of the spatial portions.
    Type: Application
    Filed: April 20, 2023
    Publication date: April 18, 2024
    Applicant: GoPro, Inc.
    Inventors: Balineedu Chowdary Adsumilli, Adeel Abbas, Sumit Chawla
  • Publication number: 20240053660
    Abstract: A camera housing includes a first surface and a second surface noncoplanar with the first surface. A first interconnect mechanism is coupled to the first surface and rotatable between a collapsed position and an extended position. In the collapsed position, protrusions of the first interconnect mechanism extend parallel to the first surface. In the extended position, the protrusions of the first interconnect mechanism extend in a perpendicular manner away from the first surface. A second interconnect mechanism is coupled to the second surface and rotatable between a collapsed position and an extended position. In the collapsed position, protrusions of the second interconnect mechanism include coplanar surfaces and extend adjacent to the second surface. In the extended position, the protrusions of the second interconnect mechanism extend in a perpendicular manner away from the second surface.
    Type: Application
    Filed: August 8, 2023
    Publication date: February 15, 2024
    Applicant: GoPro, Inc.
    Inventors: Bessy Wen-Han Liang, Nicholas D. Woodman, Nicholas Vitale, Huy Phuong Nguyen
  • Publication number: 20240053662
    Abstract: Modular lens assemblies and mounting systems are disclosed. A modular lens assembly includes a removable portion and a fixed portion. The removable portion includes a first lens stack configured to produce a near-collimated ray path or a collimated ray path. The fixed portion includes a second lens stack configured to receive the near-collimated ray path or the collimated ray path from the removable portion. The modular lens assembly may be implemented in an image capture device. An image sensor of the image capture device is positioned at an end of the modular lens assembly. The image sensor is configured to capture images based on light incident on the image sensor through the first lens stack and the second lens stack such that the light incident on an outer lens of the first lens stack is refracted through the second lens stack to the image sensor.
    Type: Application
    Filed: January 25, 2022
    Publication date: February 15, 2024
    Applicant: GoPro, Inc.
    Inventors: Jonathan Stern, Peiqian Zhao
  • Publication number: 20240040251
    Abstract: Systems, apparatus, and methods for stabilization and blending of exposures. So-called Electronic Image Stabilization (EIS) techniques use image manipulation software to compensate for camera motion. Unfortunately, existing EIS techniques cannot compensate for artifacts introduced by low shutter speed (e.g., blurs). Various embodiments of the present disclosure generate stabilized images from multiple exposures. In one exemplary embodiment, the stabilized exposures are blended using a linear sum of the color data for each pixel of the image. By stabilizing each exposure and linearly summing the light information, the camera shake can be removed, and the scene motion blur can be controlled. The stabilization and blending techniques enable a mathematical emulation of a selected shutter angle from many high-speed exposures.
    Type: Application
    Filed: October 10, 2023
    Publication date: February 1, 2024
    Applicant: GoPro, Inc.
    Inventor: David Newman
  • Publication number: 20240037793
    Abstract: Systems, apparatus, and methods for piggyback camera calibration. Existing piggybacked capture techniques use a “beauty camera” and an “action camera” to capture raw footage. The user directly applies the EIS stabilization track of a piggybacked action camera to the cinematic footage to create desired stable footage. Unfortunately, since the action camera may have been slightly offset from the cinematic video camera, the EIS stabilization data will only roughly approximate the necessary corrections. In other words, the user must manually fine tune the corrections. The disclosed embodiments use a calibration sequence to estimate a physical offset between the beauty camera and the action camera. Then, the estimated physical offset can be used to calculate an offset camera orientation for stabilizing the beauty camera. The foregoing process can be performed in-the-field before actual capture. This allows the user to check their set-up and fix any issues before capturing the desired footage.
    Type: Application
    Filed: July 27, 2022
    Publication date: February 1, 2024
    Applicant: GoPro, Inc.
    Inventors: Andrew Russell, Robert McIntosh
  • Publication number: 20240013345
    Abstract: Methods and apparatus for shared image processing among multiple devices. In one embodiment, an exemplary action camera performs a partial multiband blend. Even though the action camera may not have resources to handle the multiband blend of the entire action camera's footage, it can do a significant portion. The partially blended content can be used in ready-to-share applications, or completely blended by another device.
    Type: Application
    Filed: June 28, 2023
    Publication date: January 11, 2024
    Applicant: GoPro, Inc.
    Inventors: Gladys Jocelyne Marc, Alexis Lefebvre, Cesar Douady, Guillaume Guerin
  • Publication number: 20230410251
    Abstract: Apparatus and methods for optimized stitching of overcapture content. In one embodiment, the optimized stitching of the overcapture content includes capturing the overcapture content; producing overlap bands associated with the captured overcapture content; downsampling the produced overlap bands; generating derivative images from the downsampled overlap bands; generating a cost map associated with the generated derivative images; determining shortest path information for the generated cost map; generating a warp file based on the determined shortest path information, the generated warp file being utilized for the optimized stitching of the overcapture content. Camera apparatus and a non-transitory computer-readable apparatus are also disclosed.
    Type: Application
    Filed: August 30, 2023
    Publication date: December 21, 2023
    Applicant: GoPro, Inc.
    Inventors: Daryl Stimm, David Newman, Kyler William Schwartz, William Edward MacDonald
  • Publication number: 20230368333
    Abstract: Systems, apparatus, and methods for motion transfer between different media. Two videos shot simultaneously are combined with aspects of each used to produce a higher-resolution frame interpolated video than would otherwise be possible by the devices alone. The interpolated video may be used in slow motion footage, a higher frame rate (e.g., virtual reality) application, or to add a realistic motion blur effect in post-processing (rather than in capture). A post-processing device may receive high-speed video from a high-speed camera and a low-speed video from a low-speed camera. The post-processing device may use motion data from the high-speed video to interpolate frames of (high-resolution) low-speed video producing low-speed interpolated video.
    Type: Application
    Filed: May 11, 2022
    Publication date: November 16, 2023
    Applicant: GoPro, Inc.
    Inventor: Robert McIntosh
  • Publication number: 20230362319
    Abstract: Apparatus and methods for stitching images, or re-stitching previously stitched images. Specifically, the disclosed systems in one implementation save stitching information and/or original overlap source data during an original stitching process. During subsequent retrieval, rendering, and/or display of the stitched images, the originally stitched image can be flexibly augmented, and/or re-stitched to improve the original stitch quality. Practical applications of the disclosed solutions enable, among other things, a user to create and stitch a wide field of view (FOV) panorama from multiple source images on a device with limited processing capability (such as a mobile phone or other capture device). Moreover, post-processing stitching allows for the user to convert from one image projection to another without fidelity loss (or with an acceptable level of loss).
    Type: Application
    Filed: July 10, 2023
    Publication date: November 9, 2023
    Applicant: GoPro, Inc.
    Inventors: Adeel Abbas, Cesar Douady, Timothy MacMillan
  • Publication number: 20230351656
    Abstract: A method includes determining an altitude of a camera of an aerial vehicle, determining a field of view (FOV) of a camera, generating a localized map, determining a relative position of the aerial vehicle on the localized map, and determining a relative heading of the aerial vehicle.
    Type: Application
    Filed: July 6, 2023
    Publication date: November 2, 2023
    Applicant: GoPro, Inc.
    Inventor: Joseph Anthony Enke
  • Publication number: 20230324772
    Abstract: A camera mount, comprising: a base and a bottom. The bottom is movably connected to the base. The bottom includes recesses configured to receive flexible fasteners. The bottom and the base are movable to a closed position where the recesses are inaccessible and the flexible fasteners are held between the base and the bottom and an open position where the recesses are exposed so that the flexible fasteners are at least one of insertable, removable, or interchangeable.
    Type: Application
    Filed: April 4, 2023
    Publication date: October 12, 2023
    Applicant: GoPro, Inc.
    Inventors: Edward Gordon Russell, Bessy Wen-Han Liang, Matthew Masao Tonokawa, Jordan Zook Todd
  • Publication number: 20230328345
    Abstract: An image capture device having a housing and a heatsink. The image capture device includes an integrated sensor and lens assembly. The heatsink is located partially or completely within the housing. The heatsink includes a through which a portion of the integrated sensor and lens assembly extends, and a planar surface located adjacent to the cutout. The planar surface includes a front side and a rear side. A printed circuit board in communication with a rear side of the planar surface of the heatsink. An LCD recess located in a front side of the planar surface of the heatsink that is configured to receive a liquid crystal display, wherein the LCD recess is opposite the printed circuit board.
    Type: Application
    Filed: June 16, 2023
    Publication date: October 12, 2023
    Applicant: GoPro, Inc.
    Inventors: Nicholas Vitale, Raul Vargas Gonzalez, Herman Wong
  • Publication number: 20230281756
    Abstract: Systems, apparatus, and methods for super-resolution of non-uniform spatial blur. Non-uniform spatial blur presents unique challenges for conventional neural network processing. Existing implementations attempt to handle super-resolution with a “brute force” optimization. Various embodiments of the present disclosure subdivide the super-resolution function into sub-steps. “Unfolding” super-resolution into smaller closed-form functions allows for operation generic plug-and-play convolutional neural network (CNN) logic. Additionally, each step can be optimized with its own step-specific hyper parameters to improve performance.
    Type: Application
    Filed: March 7, 2023
    Publication date: September 7, 2023
    Applicant: GoPro, Inc.
    Inventors: Charles Laroche, Matias Tassano Ferres, Andrés Almansa
  • Publication number: 20230245688
    Abstract: Systems, apparatus, and methods for encoding telemetry data as audio waveforms. Action cameras may be used to capture telemetry data, which in some applications, may be more useful than its audio/visual media capture. For example, a user may desire the “look-and-feel” of one camera but would like to use the telemetry track information from an action camera. In so-called “piggybacking” operation, the user may rigidly affix the two cameras and use the video track with the telemetry track in post-processing. Unfortunately, time-aligning the two tracks can result in relative drift over time. Various embodiments of the present disclosure transfer telemetry data as data-over-audio. The data-over-audio may be sampled according to the recipient device and recorded at capture. Subsequent post-processing may be performed with the time-aligned tracks.
    Type: Application
    Filed: June 2, 2022
    Publication date: August 3, 2023
    Applicant: GoPro, Inc.
    Inventors: David Newman, Robert McIntosh
  • Publication number: 20230247292
    Abstract: Methods and apparatus for stabilizing image data based on a lens polynomial. Non-rectilinear footage can be captured and rectified in-camera; the rectified images may be stabilized to provide rectified stable video. In one exemplary embodiment, the footage is rectified and stabilized based on a lens polynomial and the camera's own movement. In some variants, the rectified stable video may be stored along with its margin track. In-camera rectified stable video provides several benefits over traditional techniques (e.g., the ability to share rectilinear content from the camera without additional post-processing, as well as reduced file sizes of the shared videos). Lens-aware post-processing can reuse portions of the in-camera rectified stable videos while providing additional benefits (e.g., the ability to re-frame the video in post-production).
    Type: Application
    Filed: May 31, 2022
    Publication date: August 3, 2023
    Applicant: GoPro, Inc.
    Inventors: Daryl Stimm, Kyler William Schwartz, Andrew Russell, Robert McIntosh
  • Publication number: 20230224583
    Abstract: Systems, apparatus, and methods for stabilization and blending of exposures. So-called Electronic Image Stabilization (EIS) techniques use image manipulation software to compensate for camera motion. Unfortunately, existing EIS techniques cannot compensate for artifacts introduced by low shutter speed (e.g., blurs). Various embodiments of the present disclosure generate stabilized images from multiple exposures. In one exemplary embodiment, the stabilized exposures are blended using a linear sum of the color data for each pixel of the image. By stabilizing each exposure and linearly summing the light information, the camera shake can be removed, and the subject motion blur can be controlled. The stabilization and blending techniques enable a mathematical emulation of a selected shutter angle from many high-speed exposures.
    Type: Application
    Filed: January 10, 2022
    Publication date: July 13, 2023
    Applicant: GoPro, Inc.
    Inventor: David Newman
  • Publication number: 20230215051
    Abstract: Systems, apparatus, and methods for color space representation. Unlike television and other passively viewed content, action cameras enable the user to take an active role in content generation/modification. Ideally, image content can be transferred from action cameras to post-processing devices with minimal signal loss (noise introduction), however existing device ecosystems are often bottlenecked by commodity codecs and/or intermediary networks. Various embodiments of the present disclosure use an improved “transport” compression technique to preserve desirable signal codeword and noise codeword relationships for color space representation.
    Type: Application
    Filed: January 6, 2022
    Publication date: July 6, 2023
    Applicant: GoPro, Inc.
    Inventor: David NEWMAN
  • Publication number: 20230152673
    Abstract: An accessory for an image capture device is disclosed that includes: a body; a rotatable first support that is configured for connection to the image capture device; a second support that is pivotably reconfigurable between stowed and deployed configurations; and a third support including first and second legs that are pivotably connected to the body such that the third support is reconfigurable between collapsed and expanded configurations. In the stowed configuration, the second support is concealed within the body, and in the deployed configuration, the second support extends outwardly from the body to facilitate connection of the accessory to an ancillary product. In the collapsed configuration, the third support defines a grip for the image capture device, and in the expanded configuration, the body defines a third leg that cooperates with the first and second legs to provide a freestanding base for the image capture device.
    Type: Application
    Filed: November 11, 2022
    Publication date: May 18, 2023
    Applicants: GoPro, Inc., GoPro, Inc.
    Inventors: Matthew Masao Tonokawa, Alexis De Stasio, Jesse Frost Patterson, Matthew David Thomas, Huy Phuong Nguyen
  • Publication number: 20230141884
    Abstract: Methods and apparatus for just-in-time streaming media. Existing content delivery networks are optimized for providing mass media to many consumers. This delivery model is poorly suited to user-specific content. Exemplary embodiments of the present disclosure create a program instance that can service a client's media requests from their archival data. In one specific implementation, the archival data is stored segments that are ready for streaming; a content server may provide either a consolidated file or a media “quasi-stream” from the same storage object(s). The quasi-stream supports progressive playback (media playback as it is being downloaded.) The program instance provides the client device the illusion of a static file system, however client requested access to HTTP file downloads are provided in packets that are transmuxed/transcoded from archival data.
    Type: Application
    Filed: December 29, 2022
    Publication date: May 11, 2023
    Applicant: GoPro, Inc.
    Inventors: Daniel Dennedy, Rahul Iyengar, Cedric Fernandes
  • Publication number: 20230144635
    Abstract: Methods and apparatus for metadata-based cinematography, production effects, shot selection, and/or other content augmentation. Effective cinematography conveys storyline, emotion, excitement, etc. Unfortunately, most amateur filmmakers lack the knowledge and ability to create cinema quality media. Various aspects of the present disclosure are directed to, among other things, rendering media based on instantaneous metadata. Unlike traditional post-processing techniques that rely on human subjectivity, some of the various techniques described herein leverage the camera's actual experiential data to enable cinema-quality post-processing for the general consuming public. Instantaneous metadata-based cinematography and shot selection advisories and architectures are also described.
    Type: Application
    Filed: November 29, 2022
    Publication date: May 11, 2023
    Applicant: GoPro, Inc.
    Inventor: David A. Newman
  • Publication number: 20230106957
    Abstract: Apparatus and methods for the pre-processing of image data so as to enhance quality of subsequent encoding and rendering. In one embodiment, a capture device is disclosed that includes a processing apparatus and a non-transitory computer readable apparatus comprising a storage medium have one or more instructions stored thereon. The one or more instructions, when executed by the processing apparatus, are configured to: receive captured image data (such as that sourced from two or more separate image sensors) and pre-process the data to enable stabilization of the corresponding images prior to encoding. In some implementations, the pre-processing includes combination (e.g., stitching) of the captured image data associated with the two or more sensors to facilitates the stabilization. Advantageously, undesirable artifacts such as object “jitter” can be reduced or eliminated. Methods and non-transitory computer readable apparatus are also disclosed.
    Type: Application
    Filed: December 8, 2022
    Publication date: April 6, 2023
    Applicant: GoPro, Inc.
    Inventor: David Newman
  • Publication number: 20230109047
    Abstract: Methods and apparatus for post-processing in-camera stabilized video. Embodiments of the present disclosure reconstruct and re-stabilize an in-camera stabilized video to provide for improved stabilization (e.g., a wider crop, etc.) In-camera sensor data may be stored and used to re-calculate orientation metadata in post-production. In-camera stabilization provides several benefits (e.g., the ability to share stabilized videos from the camera without additional post-processing as well as reduced file sizes of the shared videos). Camera-aware post-processing can reuse portions of the in-camera stabilized videos while providing additional benefits (e.g., the ability to regenerate the original captured videos in post-production and re-stabilize the videos). Camera-aware post-processing can also improve orientation metadata and remove sensor error. The disclosed techniques also enable assisted optical flow-based stabilization using the refined metadata.
    Type: Application
    Filed: October 1, 2021
    Publication date: April 6, 2023
    Applicant: GoPro, Inc.
    Inventors: Daryl Stimm, Andrew Russell, Robert McIntosh
  • Publication number: 20230051215
    Abstract: A method and device are provided for positioning a mounted camera. The device includes a holding element that secures the mounted camera to the device, a wireless linkage at which remote attitude commands representing attitude changes of a remote driver are received, a local controller that interprets the remote attitude commands and generates local attitude commands that move the camera to mimic an orientation of the remote driver, and an attitude sensing element that senses a local attitude of the device. The attitude sensing element includes a gyro, an accelerometer, or a magnetometer, and jitter present in the remote attitude commands is removed and not passed on to the local attitude commands.
    Type: Application
    Filed: October 31, 2022
    Publication date: February 16, 2023
    Applicant: GoPro, Inc.
    Inventors: Josh Guyot, JoeBen Bevirt
  • Publication number: 20230009943
    Abstract: Apparatus and methods for stitching images, or re-stitching previously stitched images. Specifically, the disclosed systems in one implementation save stitching information and/or original overlap source data during an original stitching process. During subsequent retrieval, rendering, and/or display of the stitched images, the originally stitched image can be flexibly augmented, and/or re-stitched to improve the original stitch quality. Practical applications of the disclosed solutions enable, among other things, a user to create and stitch a wide field of view (FOV) panorama from multiple source images on a device with limited processing capability (such as a mobile phone or other capture device). Moreover, post-processing stitching allows for the user to convert from one image projection to another without fidelity loss (or with an acceptable level of loss).
    Type: Application
    Filed: September 15, 2022
    Publication date: January 12, 2023
    Applicant: GoPro, Inc.
    Inventors: Adeel Abbas, Cesar Douady, Timothy MacMillan
  • Publication number: 20230011843
    Abstract: Methods and apparatus for image processing of spherical content via hardware acceleration components. In one embodiment, an EAC image is subdivided into facets via existing software addressing and written into the memory buffers (normally used for rectilinear cubemaps) in a graphics processing unit (GPU). The EAC facets may be translated, rotated, and/or mirrored so as to align with the expected three-dimensional (3D) coordinate space. The GPU may use existing hardware accelerator logic, parallelization, and/or addressing logic to greatly improve 3D image processing effects (such as a multi-band blend using Gaussian blurs.
    Type: Application
    Filed: September 8, 2022
    Publication date: January 12, 2023
    Applicant: GoPro, Inc.
    Inventors: William Edward MacDonald, Kyler William Schwartz, David A. Newman
  • Publication number: 20220385811
    Abstract: Methods and apparatus for shear correction in spherical projections. Embedded devices generally lack the compute and/or memory resources to implement two-dimensional (2D) stitches for spherical projections. Objects (such as a mounting fixture) within a certain distance of the camera may experience a 2D “tear” or “shear” artifact when stitched. Various embodiments of the present disclosure perform two orthogonal 1D stitches: (i) latitudinally across the meridian and (ii) longitudinally along the meridian to approximate the effect of a 2D stitch. Notably, the 1D stitch may be less precise than a true 2D stitch, however the image portion being stitched (e.g., the camera mount) is not the user's subject of interest and can be rendered much more loosely without adverse consequence. Temporal smoothing optimizations are additionally disclosed.
    Type: Application
    Filed: August 10, 2022
    Publication date: December 1, 2022
    Applicant: GoPro, Inc.
    Inventors: David Newman, Daryl Stimm
  • Publication number: 20220368746
    Abstract: Methods and apparatus for just-in-time streaming media. Existing content delivery networks are optimized for providing mass media to many consumers. This delivery model is poorly suited to user-specific content. Exemplary embodiments of the present disclosure create a program instance that can service a client's media requests from their archival data. In one specific implementation, the archival data is stored segments that are ready for streaming; a content server may provide either a consolidated file or a media “quasi-stream” from the same storage object(s). The quasi-stream supports progressive playback (media playback as it is being downloaded.) The program instance provides the client device the illusion of a static file system, however client requested access to HTTP file downloads are provided in packets that are transmuxed/transcoded from archival data.
    Type: Application
    Filed: May 14, 2021
    Publication date: November 17, 2022
    Applicant: GoPro, Inc.
    Inventors: Daniel Dennedy, Rahul Iyengar, Cedric Fernandes
  • Publication number: 20220108559
    Abstract: A face located along a stitch line in a spherical image is detected by rendering views of regions of the spherical image along the stitch line. The spherical image may be produced by combining first and second images. A first view of a projection of the spherical image is rendered. A scaling factor for rendering a second view of the projection is determined based characteristics of the first portion of the face. The second view is then rendered according to the scaling factor. The use of the scaling factor to render the second view causes a change in the depiction of the second portion of the face. For example, the scaling factor can indicate to change the resolution or expected size of the second portion of the face when rendering the second view. A face is then detected within the spherical image based on the rendered first and second views.
    Type: Application
    Filed: January 2, 2020
    Publication date: April 7, 2022
    Applicant: GoPro, Inc.
    Inventors: Thomas Armel Derbanne, Bruno César Douady, Maxim Karpushin
  • Publication number: 20220092729
    Abstract: Hyper-hemispherical images may be combined to generate a rectangular projection of a spherical image having an equatorial stitch line along of a line of lowest distortion in the two images. First and second circular images are received representing respective hyper-hemispherical fields of view. A video processing device may project each circular image to a respective rectangular image by mapping an outer edge of the circular image to a first edge of the rectangular image and mapping a center point of the circular image to a second edge of the first rectangular image. The rectangular images may be stitched together along the edges corresponding to the outer edge of the original circular image.
    Type: Application
    Filed: December 6, 2021
    Publication date: March 24, 2022
    Applicant: GoPro, Inc.
    Inventors: Joseph Steel, Timothy Macmillan
  • Publication number: 20220060738
    Abstract: Methods and apparatus for processing of video content to optimize codec bandwidth. In one embodiment, the method includes capturing panoramic imaging content (e.g., a 360° panorama), mapping the panoramic imaging content into an equi-angular cubemap (EAC) format, and splitting the EAC format into segments for transmission to maximize codec bandwidth. In one exemplary embodiment, the EAC segments are transmitted at a different frame rate than the subsequent display rate of the panoramic imaging content. For example, the mapping and frame rate may be chosen to enable the rendering of 8K, 360° content at 24 fps, using commodity encoder hardware and software that nominally supports 4K content at 60 fps.
    Type: Application
    Filed: November 2, 2021
    Publication date: February 24, 2022
    Applicant: GoPro, Inc.
    Inventors: David A. Newman, Cesar Douady, Alexis Lefebvre, Daryl Stimm, Michael Kraak
  • Publication number: 20220044357
    Abstract: Apparatus and methods for optimized stitching of overcapture content. In one embodiment, the optimized stitching of the overcapture content includes capturing the overcapture content; producing overlap bands associated with the captured overcapture content; downsampling the produced overlap bands; generating derivative images from the downsampled overlap bands; generating a cost map associated with the generated derivative images; determining shortest path information for the generated cost map; generating a warp file based on the determined shortest path information, the generated warp file being utilized for the optimized stitching of the overcapture content. Camera apparatus and a non-transitory computer-readable apparatus are also disclosed.
    Type: Application
    Filed: October 25, 2021
    Publication date: February 10, 2022
    Applicant: GoPro, Inc.
    Inventors: Daryl Stimm, David Newman, Kyler William Schwartz, William Edward MacDonald
  • Publication number: 20210392375
    Abstract: Methods and apparatus for processing of video content to optimize codec bandwidth. In one embodiment, the method includes capturing panoramic imaging content (e.g., a 360° panorama), mapping the panoramic imaging content into an equi-angular cubemap (EAC) format, and splitting the EAC format into segments for transmission to maximize codec bandwidth. In one exemplary embodiment, the EAC segments are transmitted at a different frame rate than the subsequent display rate of the panoramic imaging content. For example, the mapping and frame rate may be chosen to enable the rendering of 8K, 360° content at 24 fps, using commodity encoder hardware and software that nominally supports 4K content at 60 fps.
    Type: Application
    Filed: August 30, 2021
    Publication date: December 16, 2021
    Applicant: GoPro, Inc.
    Inventors: David A. Newman, Cesar Douady, Alexis Lefebvre, Daryl Stimm, Michael Kraak
  • Publication number: 20210385473
    Abstract: A system and method disposed to enable encoding, decoding and manipulation of digital video with substantially less processing load than would otherwise be required. In particular, one disclosed method is directed to generating a compressed video data structure that is selectively decodable to a plurality of resolutions including the full resolution of the uncompressed stream. The desired number of data components and the content of the data components that make up the compressed video data, which determine the available video resolutions, are variable based upon the processing carried out and the resources available to decode and process the data components. During decoding, efficiency is substantially improved because only the data components necessary to generate a desired resolution are decoded. In variations, both temporal and spatial decoding are utilized to reduce frame rates, and hence, further reduce processor load.
    Type: Application
    Filed: August 12, 2021
    Publication date: December 9, 2021
    Applicant: GoPro, Inc.
    Inventors: David A. Newman, Brian G. Schunck
  • Publication number: 20210327026
    Abstract: Methods and apparatus for blending unknown pixels in overlapping images. In one embodiment, an action camera captures two hyper-hemispherical fisheye images that are stitched to a 360° panorama. In order to remove exposure differences between the two cameras, the images are pre-processed prior to multiband blending. The pre-processing leverages image information from pixels to make informed guesses about pixels that were not captured. In particular, various pixels with different knowability (e.g., known, unknown, consistent, and conflicting) may be handled differently so as to emphasize/de-emphasize their importance in pre-processing.
    Type: Application
    Filed: April 14, 2021
    Publication date: October 21, 2021
    Applicant: GoPro, Inc.
    Inventor: Marc Lebrun
  • Publication number: 20210321047
    Abstract: Apparatus and methods for stitching images, or re-stitching previously stitched images. Specifically, the disclosed systems in one implementation save stitching information and/or original overlap source data during an original stitching process. During subsequent retrieval, rendering, and/or display of the stitched images, the originally stitched image can be flexibly augmented, and/or re-stitched to improve the original stitch quality. Practical applications of the disclosed solutions enable, among other things, a user to create and stitch a wide field of view (FOV) panorama from multiple source images on a device with limited processing capability (such as a mobile phone or other capture device). Moreover, post-processing stitching allows for the user to convert from one image projection to another without fidelity loss (or with an acceptable level of loss).
    Type: Application
    Filed: April 26, 2021
    Publication date: October 14, 2021
    Applicant: GoPro, Inc.
    Inventors: Adeel Abbas, Cesar Douady, Timothy MacMillan
  • Publication number: 20210272759
    Abstract: A button is disclosed for use with an image capture device in an underwater environment. The button includes a movable plunger configured to cause actuation of the image capture device, and upper and lower components collectively defining an internal cavity that is configured to receive the plunger. The upper and lower components are configured and connected such that actuation of the image capture device is prevented until a threshold pressure is applied to the upper component that is greater than external water pressure applied in the underwater environment. The upper component includes at least one opening that is configured to allow water to enter the internal cavity to modify an internal pressure within the internal cavity so as to reduce the threshold pressure required to actuate the image capture device in the underwater environment.
    Type: Application
    Filed: July 3, 2019
    Publication date: September 2, 2021
    Applicant: GoPro, Inc.
    Inventors: Chao Chen, Marco Marroquin, Himay Shukla
  • Publication number: 20210250507
    Abstract: Apparatus and methods for the pre-processing of image data so as to enhance quality of subsequent encoding and rendering. In one embodiment, a capture device is disclosed that includes a processing apparatus and a non-transitory computer readable apparatus comprising a storage medium have one or more instructions stored thereon. The one or more instructions, when executed by the processing apparatus, are configured to: receive captured image data (such as that sourced from two or more separate image sensors) and pre-process the data to enable stabilization of the corresponding images prior to encoding. In some implementations, the pre-processing includes combination (e.g., stitching) of the captured image data associated with the two or more sensors to facilitates the stabilization. Advantageously, undesirable artifacts such as object “jitter” can be reduced or eliminated. Methods and non-transitory computer readable apparatus are also disclosed.
    Type: Application
    Filed: April 26, 2021
    Publication date: August 12, 2021
    Applicant: GoPro, Inc.
    Inventor: David Newman
  • Publication number: 20210203844
    Abstract: Methods and apparatus for shear correction in spherical projections. Embedded devices generally lack the compute and/or memory resources to implement two-dimensional (2D) stitches for spherical projections. Objects (such as a mounting fixture) within a certain distance of the camera may experience a 2D “tear” or “shear” artifact when stitched. Various embodiments of the present disclosure perform two orthogonal 1D stitches: (i) latitudinally across the meridian and (ii) longitudinally along the meridian to approximate the effect of a 2D stitch. Notably, the 1D stitch may be less precise than a true 2D stitch, however the image portion being stitched (e.g., the camera mount) is not the user's subject of interest and can be rendered much more loosely without adverse consequence. Temporal smoothing optimizations are additionally disclosed.
    Type: Application
    Filed: December 31, 2020
    Publication date: July 1, 2021
    Applicant: GoPro, Inc.
    Inventors: David Newman, Daryl Stimm
  • Publication number: 20210084200
    Abstract: An image capture device may include an image sensor, a processor, and a memory. The image sensor may be configured to obtain an image. The processor may be configured to generate a grid on the image. The grid may include one or more vertices. The one or more vertices may be used to form tiles. The processor may be configured to determine a flare level of each vertex. The processor may be configured to assign a maximum flare level for each tile of the image. The processor may be configured to sort the tiles. The tiles may be sorted based on the maximum flare level of each tile. The processor may be configured to apply a flare compensation to a subset of the tiles to obtain a processed image. The processed image may have reduced flare artifacts or no flare artifacts. The processed image may be stored in the memory.
    Type: Application
    Filed: August 24, 2020
    Publication date: March 18, 2021
    Applicant: GoPro, Inc.
    Inventors: Bruno César Douady, Guillaume Matthieu Guérin, Eric Chasseur
  • Publication number: 20200267339
    Abstract: Systems and methods are disclosed for image signal processing. For example, methods may include receiving a current image of a sequence of images from an image sensor; combining the current image with a recirculated image to obtain a noise reduced image, where the recirculated image is based on one or more previous images of the sequence of images from the image sensor; determining a noise map for the noise reduced image, where the noise map is determined based on estimates of noise levels for pixels in the current image, a noise map for the recirculated image, and a set of mixing weights; recirculating the noise map with the noise reduced image to combine the noise reduced image with a next image of the sequence of images from the image sensor; and storing, displaying, or transmitting an output image that is based on the noise reduced image.
    Type: Application
    Filed: May 24, 2017
    Publication date: August 20, 2020
    Applicants: GoPro, Inc., GoPro, Inc.
    Inventors: Bruno César Douady-Pleven, Michael Serge Andre Kraak, Guillaume Matthieu Guérin, Thomas Nicolas Emmanuel Veit
  • Publication number: 20190177002
    Abstract: A condition of an unmanned aerial vehicle (UAV) is detected using one or more sensors of the UAV and signaled according to an alert definition associated with the condition. For example, an alert definition can indicate to signal the condition by using a motor of the UAV to produce an audible tone. A tonal signal having a frequency within an audible spectrum can be generated according to the alert definition. The tonal signal and a drive signal used for supplying current to the motor can be combined to produce a combined signal. The combined signal can then be transmitted to the motor to cause the motor to produce the audible tone. In some cases, an amplitude of the tonal signal can be modulated, such as where the amplitude of the combined signal exceeds a threshold associated with an operating margin of the UAV.
    Type: Application
    Filed: December 7, 2017
    Publication date: June 13, 2019
    Applicant: GoPro, Inc.
    Inventors: Joseph A. Enke, Miles Hansen, Mehran Ferdowsi
  • Publication number: 20180316843
    Abstract: A method and device are provided for positioning a mounted camera. The device includes a holding element that secures the mounted camera to the device, a wireless linkage at which remote attitude commands representing attitude changes of a remote driver are received, a local controller that interprets the remote attitude commands and generates local attitude commands that move the camera to mimic an orientation of the remote driver, and an attitude sensing element that senses a local attitude of the device. The attitude sensing element includes a gyro, an accelerometer, or a magnetometer, and jitter present in the remote attitude commands is removed and not passed on to the local attitude commands.
    Type: Application
    Filed: July 9, 2018
    Publication date: November 1, 2018
    Applicant: GoPro, Inc.
    Inventors: Josh Guyot, JoeBen Bevirt
  • Publication number: 20180255241
    Abstract: System and methods are disclosed for generating an output video stream from a wide field video stream. The wide field video stream may be created from compiling a plurality of video stream capturing different viewpoints of the same environment simultaneously. Upon selecting a wide field video stream, a first geometric function may be utilized to generate individual intermediate points that correspond to locations in a wide field video stream to be displayed on a particular display space. A second geometric function may be further utilized to generate reference points so that a the pixel on the output video stream are determined from the intermediate points of the wide field video stream of the first geometric function. Upon the execution of the first and second geometric function, an output video stream is generated for display on the determined display space.
    Type: Application
    Filed: October 19, 2015
    Publication date: September 6, 2018
    Applicant: GOPRO, INC.
    Inventors: Alexandre JENNY, Yann Renaud GILQUIN
  • Publication number: 20170323241
    Abstract: An automated system to receive requirements of users and present them competitive products and services for selection, authorization, order placement, supplier invoicing and payment, all through a single system. The system brings together a number of tools to deliver the functionalities required to select most appropriate products and services from a plurality of supplier databases and may also incorporate user inputs as required. The system enables cost savings, manages suppliers, reduces costs of ownership, enhances catalog management, optimizes suppliers, enhances buyer experience and obtains legally compliant agreements with suppliers.
    Type: Application
    Filed: April 24, 2017
    Publication date: November 9, 2017
    Applicant: GoProcure, Inc.
    Inventors: Sandeep Gauba, Roy R. Anderson, Amit Jhamvar
  • Publication number: 20170110151
    Abstract: Videos may be automatically generated using a set of video clip. Individual moments of interest may be identified within individual video clips of a set of video clips. A moment of interest may correspond to a point in time within a video clip. The point in time may be associated with one or more values of one or more attributes of the video clip. Individual moments of interest may be associated with individual portions of a video. The video may be generated using the set of video clips based on the associations.
    Type: Application
    Filed: October 20, 2015
    Publication date: April 20, 2017
    Applicant: GOPRO, INC.
    Inventors: Joven Matias, Ha Phan
  • Publication number: 20170109585
    Abstract: Users desiring to generate videos from video clips may want to locate moments of interest within the video clips. A system and method described herein may be configured to provide recommendations of moments of interest within video clips post capture of the video clips. User accounts associated with users of the system may include preference information that defines user preferences with respect to values of attributes of video clips. Moments of interest may be identified within individual video clips when the individual video clips have at least one value of at least one attribute specified by the user preferences. Recommendations of identified moments of interest may be provide to users.
    Type: Application
    Filed: October 20, 2015
    Publication date: April 20, 2017
    Applicant: GOPRO, INC.
    Inventors: Joven Matias, Ha Phan
  • Publication number: 20160112635
    Abstract: System and methods are disclosed for generating an output video stream from a wide field video stream. The wide field video stream may be created from compiling a plurality of video stream capturing different viewpoints of the same environment simultaneously. Upon selecting a wide field video stream, a first geometric function may be utilized to generate individual intermediate points that correspond to locations in a wide field video stream to be displayed on a particular display space. A second geometric function may be further utilized to generate reference points so that a the pixel on the output video stream are determined from the intermediate points of the wide field video stream of the first geometric function. Upon the execution of the first and second geometric function, an output video stream is generated for display on the determined display space.
    Type: Application
    Filed: October 19, 2015
    Publication date: April 21, 2016
    Applicant: GOPRO, INC.
    Inventors: Alexandre JENNY, Yann Renaud GILQUIN
  • Publication number: 20160088222
    Abstract: System and methods are disclosed for stitching a plurality of video streams to generate a wide field video stream. The wide field video stream may be created by obtaining multiple video streams that correspond to a common period in time at which portion of the multiple video streams were captured. A reference instant the pertains to a common period of time order may be determined to help stitch the images corresponding to the video streams in chronological order. A reference value is then calculated for a construction parameter of one or more images from the multiple video streams captured at times that correspond to the determined reference instant. A panoramic image is then constructed by stitching together the images that correspond to the determined reference instant, thus further generating a wide field video stream.
    Type: Application
    Filed: December 2, 2015
    Publication date: March 24, 2016
    Applicant: GOPRO, INC.
    Inventors: Alexandre Jenny, Renan Coudray
  • Publication number: 20160037068
    Abstract: Video streams may be stitched together to form a single wide field video stream. The wide field video stream may exhibit a view with a field angle that may be larger than a field angled of an individual video stream. The wide field video stream may exhibit, for example, panoramic views. A method of stitching together video stream may comprise one or more of determining at least one reference time instance within a reference video stream; determining a first set of values of parameters used to generate a first panoramic image that comprises a combination of images that correspond to the at least one reference time instance; generating panoramic images that comprise images of individual video streams based on the first set of values of the parameters; and/or other operations.
    Type: Application
    Filed: October 12, 2015
    Publication date: February 4, 2016
    Applicant: GOPRO, INC.
    Inventors: Alexandre JENNY, Renan COUDRAY
  • Publication number: 20150326748
    Abstract: A camera is configured with multiple microphones to reduce wind noise in audio data collected by the camera. The camera receives motion data, which may comprise data indicating acceleration of the camera, a plurality of video frames received by the camera, or a background level of noise associated with one or more microphones configured on the camera. The camera determines a motion vector from the motion data. The motion vector is parallel to the direction of motion of the camera. The camera selects a subset of one or more microphones in the direction of the motion vector. By recording audio data using the one or more selected microphones, the camera reduces wind noise in the audio data.
    Type: Application
    Filed: May 12, 2014
    Publication date: November 12, 2015
    Applicant: GoPro, Inc.
    Inventors: Erich Tisch, Fabrice Barbier, Scott Patrick Campbell