Abstract: The present disclosure describes systems and methods for audio/visual production. An example method includes capturing first video/audio during an initial video/audio capture session. The method also includes, during the initial video/audio capture session, determining at least one time stamp for each word spoken in the first video/audio. The method further includes, in response to receiving a retake request, selecting a cut point and a cut word. The cut word is spoken during or adjacent to the cut point. The method includes capturing second video/audio during a subsequent video/audio capture session. The method yet further includes cutting and merging a first clip from the first video/audio and a second clip from the second video/audio based on the cut point and the at least one time stamp corresponding to the cut word.

Abstract: The present disclosure relates to systems and methods for temporally aligning media elements. Example methods include providing an audio input waveform based on an audio input and receiving a text input. The example method also includes converting the text input to a text-to-speech input waveform and extracting, with an audio feature extractor, characteristic audio features from the audio input waveform and the text-to-speech input waveform. The example method yet further includes comparing audio input waveform features and text-to-speech waveform features and, based on the comparison, temporally aligning a displayed version of the text input with the audio input.

Abstract: Imaging systems can often gather higher quality information about a field of view than the unaided human eye. For example, telescopes may magnify very distant objects, microscopes may magnify very small objects, and high frame-rate cameras may capture fast motion. The present disclosure includes devices and methods that provide real-time vision enhancement without the delay of replaying from storage media. The disclosed devices and methods may include a live view user interface with two or more interactive features or effects that may be controllable in real-time. Specifically, the disclosed devices and methods may include a live view display and image and other information enhancements, which utilize in-line computation and constant control.

Abstract: The present disclosure relates to methods and systems for providing a visual teleport window. An example system includes a wide-angle camera, a display, and a controller. The controller includes at least one processor and a memory. The at least one processor executes instructions stored in the memory so as to carry out operations. The operations include receiving remote viewport information. The viewport information is indicative of a relative location of at least one eye of a remote user with respect to a remote display. The operations also include causing the wide-angle camera to capture an image of an environment of the system. The operations additionally include, based on the viewport information and information about the remote display, cropping and projecting the image to form a frame. The operations also include transmitting the frame for display at the remote display.

Abstract: The present disclosure relates to systems and methods for streaming a tiled representation of video content based on a viewing direction and desired zoom level. Example methods include receiving information indicative of a view direction and a zoom level. The view direction and the zoom level correspond to a field of view including a first portion of the video content. A second portion of the video content is outside of the field of view. The method also includes selecting, from a plurality of viewports, an active viewport corresponding to the first portion of the video content. The active viewport specifies a set of one or more active tiles of a tiled representation of the video content. The method additionally includes comparing the set of one or more active tiles with a set of one or more previous active tiles and causing streaming and rendering to stop with respect to previous active tiles that are not in the set of one or more active tiles.

Abstract: The present disclosure relates to imaging systems and methods that include a plurality of cameras configured to capture video image data based on respective fields of view of an environment. Each camera of the plurality of cameras is communicatively coupled to neighbor cameras of the plurality of cameras via a communication interface. Each camera may carry out operations include capturing video image data of the respective field of view and determining an overlay region. The overlay region includes an overlapping portion of video image data captured by the respective camera and at least one of the neighbor cameras. The operations also include cropping and warping the captured video image data of the respective field of view based on the overlay region to form respective processed video image data. The processed video image data may be uploaded to a cloud server and provided, via multiple trunk links, to a client device.

Abstract: Systems, methods, and media for providing interactive refocusing are provided, the systems comprising: a lens; an image sensor; and a processor that: causes the image sensor to capture a plurality of images over a predetermined period of time, wherein each of the plurality of images represents a scene at a different point in time; changes a depth of field between at least a pair of the plurality of images; concatenates the plurality of images to create a duration focal volume in the order in which the images were captured; computes a space-time in-focus image that represents in-focus portions from each of the plurality of images based on the duration focal volume; and computes a space-time index map that identifies an in-focus image for each location of the scene from among the plurality of images based on the duration focal volume and the space-time in-focus image.

Abstract: Systems, methods, and media for providing interactive refocusing are provided, the systems comprising: a lens; an image sensor; and a processor that: causes the image sensor to capture a plurality of images over a predetermined period of time, wherein each of the plurality of images represents a scene at a different point in time; changes a depth of field between at least a pair of the plurality of images; concatenates the plurality of images to create a duration focal volume in the order in which the images were captured; computes a space-time in-focus image that represents in-focus portions from each of the plurality of images based on the duration focal volume; and computes a space-time index map that identifies an in-focus image for each location of the scene from among the plurality of images based on the duration focal volume and the space-time in-focus image.

Abstract: The present disclosure relates to systems and methods that employ a mechanical method for adjusting camera field angles at a high speed. The adjustments may be synchronized to a high speed image capture process. As such, multiple frames can be captured without experiencing significant object movement or hand shake. The systems and methods may be used to capture videos and/or photo stitching, because both camera position and objects in the scene are static during the high speed image capture.

Abstract: The present disclosure relates to systems and methods for streaming a tiled representation of video content based on a viewing direction and desired zoom level. Example methods include receiving information indicative of a view direction and a zoom level. The view direction and the zoom level correspond to a field of view including a first portion of the video content. A second portion of the video content is outside of the field of view. The method also includes selecting, from a plurality of viewports, an active viewport corresponding to the first portion of the video content. The active viewport specifies a set of one or more active tiles of a tiled representation of the video content. The method additionally includes comparing the set of one or more active tiles with a set of one or more previous active tiles and causing streaming and rendering to stop with respect to previous active tiles that are not in the set of one or more active tiles.

Abstract: The present disclosure relates to systems and methods that deliver 360° video streams, which may be used in virtual reality and augmented reality content. Namely, a 360° video stream may be divided into a plurality of video streams, each stream of which corresponds to a portion of a viewing volume or viewing sphere. Additionally, the original 360° video stream may be downsampled to a lower resolution 360° video stream. For a given user view angle, one or more of the plurality of video streams may be considered active streams and connected to a user interface or display for data download. In such a scenario, a user interface may be connected to at least one high resolution video stream that corresponds to the user view angle as well as a low resolution 360° video stream.

Abstract: The present disclosure relates to methods and systems for providing virtual reality video content. An example method includes capturing light field image data with a plurality of cameras. The light field image data includes a plurality of sample data points. The method includes determining a viewpoint position and a viewpoint pose. The method further includes determining a nearest neighbor set based on the sample data points, the viewpoint position, and the viewpoint pose. The method also includes interpolating within the nearest neighbor set so as to form a set of resampled data points. The method yet further includes rendering a 360° image from the resampled data points. The 360° image includes a representation of the light field based on the viewpoint position and the viewpoint pose.

Abstract: Imaging systems can often gather higher quality information about a field of view than the unaided human eye. For example, telescopes may magnify very distant objects, microscopes may magnify very small objects, and high frame-rate cameras may capture fast motion. The present disclosure includes devices and methods that provide real-time vision enhancement without the delay of replaying from storage media. The disclosed devices and methods may include a live view user interface with two or more interactive features or effects that may be controllable in real-time. Specifically, the disclosed devices and methods may include a live view display and image and other information enhancements, which utilize in-line computation and constant control.

Abstract: The present disclosure relates to methods and systems for providing virtual reality video content. An example system may include a display and a sensor configured to detect a user input and a media server configured to execute instructions stored in memory so as to carry out operations. Operations include loading a nonlinear video structure. The nonlinear video structure includes a plurality of uniform resource identifiers. Each uniform resource identifier is associated with a respective video trunk. The nonlinear video structure includes an arrangement of respective video trunks coupled by at least one transition trunk. The operations include determining an initial playlist based on the nonlinear video structure, streaming the initial playlist from a media server via network, and rendering video frames to a display. The operations include, while loading the at least one transition trunk, receiving the user input and playing a next playlist based on the received user input.

Abstract: The present disclosure relates to imaging systems and methods that include a plurality of cameras configured to capture video image data based on respective fields of view of an environment. Each camera of the plurality of cameras is communicatively coupled to neighbor cameras of the plurality of cameras via a communication interface. Each camera may carry out operations include capturing video image data of the respective field of view and determining an overlay region. The overlay region includes an overlapping portion of video image data captured by the respective camera and at least one of the neighbor cameras. The operations also include cropping and warping the captured video image data of the respective field of view based on the overlay region to form respective processed video image data. The processed video image data may be uploaded to a cloud server and provided, via multiple trunk links, to a client device.

Abstract: Imaging systems can often gather higher quality information about a field of view than the unaided human eye. For example, telescopes may magnify very distant objects, microscopes may magnify very small objects, and high frame-rate cameras may capture fast motion. The present disclosure includes devices and methods that provide real-time vision enhancement without the delay of replaying from storage media. The disclosed devices and methods may include a live view user interface with two or more interactive features or effects that may be controllable in real-time. Specifically, the disclosed devices and methods may include a live view display and image and other information enhancements, which utilize in-line computation and constant control.

Abstract: A plurality of images of a scene may be obtained. These images may have been captured by an image sensor, and may include a first image and a second image. A particular gain may have been applied to the first image. An effective color temperature and a brightness of a first pixel in the first image may be determined, and a mapping between pixel characteristics and noise deviation of the image sensor may be selected. The pixel characteristics may include pixel brightness. The selected mapping may be used to map at least the brightness of the first pixel to a particular noise deviation. The brightness of the first pixel and the particular noise deviation may be compared to a brightness of a second pixel of the second image. The comparison may be used to determine whether to merge the first pixel and the second pixel.

Abstract: The present disclosure relates to systems and methods that deliver 360° video streams, which may be used in virtual reality and augmented reality content. Namely, a 360° video stream may be divided into a plurality of video streams, each stream of which corresponds to a portion of a viewing volume or viewing sphere. Additionally, the original 360° video stream may be downsampled to a lower resolution 360° video stream. For a given user view angle, one or more of the plurality of video streams may be considered active streams and connected to a user interface or display for data download. In such a scenario, a user interface may be connected to at least one high resolution video stream that corresponds to the user view angle as well as a low resolution 360° video stream.

Abstract: The present disclosure relates to systems and methods that employ a mechanical method for adjusting camera field angles at a high speed. The adjustments may be synchronized to a high speed image capture process. As such, multiple frames can be captured without experiencing significant object movement or hand shake. The systems and methods may be used to capture videos and/or photo stitching, because both camera position and objects in the scene are static during the high speed image capture.

Abstract: A plurality of images of a scene may be obtained. These images may have been captured by an image sensor, and may include a first image and a second image. A particular gain may have been applied to the first image. An effective color temperature and a brightness of a first pixel in the first image may be determined, and a mapping between pixel characteristics and noise deviation of the image sensor may be selected. The pixel characteristics may include pixel brightness. The selected mapping may be used to map at least the brightness of the first pixel to a particular noise deviation. The brightness of the first pixel and the particular noise deviation may be compared to a brightness of a second pixel of the second image. The comparison may be used to determine whether to merge the first pixel and the second pixel.