Abstract: In accordance with some embodiments, a method is performed at an image processing device with a processor and non-transitory memory. The method includes triggering light emission, over a first emission duration, having a characterizing intensity as a function of time. The method further includes obtaining respective pixel events, from an event camera, corresponding to reflections of the light emission during the first emission duration, each respective pixel event corresponding to a breach of a respective comparator threshold indicative of a brightness level, each respective pixel event characterized by a respective electrical threshold value and a timestamp at which the respective electrical threshold value was breached.

Abstract: In some embodiments, a method is performed at a device with a processor, non-transitory memory, and an event camera including pixel sensors distributed across an area. The method includes converting an event stream from a pixel sensor over a first time period into event frames by dividing the first time period into sub-periods, and binning pixel events of the event stream, where each of the sub-periods is associated with a frame sub-period identifier. The method further includes addressing the pixel sensors by sub-dividing the area into tiles, where each of the tiles includes a grouping of the pixel sensors, and a tile address of a particular pixel sensor is a combination of a tile identifier and a position locator of the particular pixel sensor. The method further includes encoding the pixel events as a function of a tile address, a frame sub-period identifier, and a brightness indicator value.

Abstract: Fiducial patterns that produce 2D Barker code-like diffraction patterns at a camera sensor are etched or otherwise provided on a cover glass in front of a camera. 2D Barker code kernels, when cross-correlated with the diffraction patterns captured in images by the camera, provide sharp cross-correlation peaks. Misalignment of the cover glass with respect to the camera can be derived by detecting shifts in the location of the detected peaks with respect to calibrated locations. Devices that include multiple cameras behind a cover glass with one or more fiducials on the cover glass in front of each camera are also described. The diffraction patterns caused by the fiducials at the various cameras may be analyzed to detect movement or distortion of the cover glass in multiple degrees of freedom.

Abstract: An electronic device may have a camera and a display. The display may be configured to display virtual reality content for a user in which no real-world content from the camera is displayed or mixed reality content in which a combination of real-world content from the camera and overlaid virtual reality content is displayed. Control circuitry in the device may adjust the display and camera while transitioning between virtual reality and mixed reality modes. The control circuitry may reconfigure the camera to exhibit a desired frame rate immediately upon transitioning from virtual reality mode to mixed reality mode. Transitions between modes may be accompanied by smooth transitions between frame rates to avoid visible artifacts on the display. The camera frame rate may be synchronized to the display frame rate for at least part of the transition between the virtual reality and mixed reality modes.

Abstract: An electronic device may have a camera and a display. The display may be configured to display virtual reality content for a user in which no real-world content from the camera is displayed or mixed reality content in which a combination of real-world content from the camera and overlaid virtual reality content is displayed. Control circuitry in the device may adjust the display and camera while transitioning between virtual reality and mixed reality modes. The control circuitry may reconfigure the camera to exhibit a desired frame rate immediately upon transitioning from virtual reality mode to mixed reality mode. Transitions between modes may be accompanied by smooth transitions between frame rates to avoid visible artifacts on the display. The camera frame rate may be synchronized to the display frame rate for at least part of the transition between the virtual reality and mixed reality modes.

Abstract: In some embodiments, a method is performed at a device with a processor, non-transitory memory, and an event camera including pixel sensors distributed across an area. The method includes converting an event stream from a pixel sensor over a first time period into event frames by dividing the first time period into sub-periods, and binning pixel events of the event stream, where each of the sub-periods is associated with a frame sub-period identifier. The method further includes addressing the pixel sensors by sub-dividing the area into tiles, where each of the tiles includes a grouping of the pixel sensors, and a tile address of a particular pixel sensor is a combination of a tile identifier and a position locator of the particular pixel sensor. The method further includes encoding the pixel events as a function of a tile address, a frame sub-period identifier, and a brightness indicator value.

Abstract: In accordance with some embodiments, a method is performed at an image processing device with a processor and non-transitory memory. The method includes triggering light emission, over a first emission duration, having a characterizing intensity as a function of time. The method further includes obtaining respective pixel events, from an event camera, corresponding to reflections of the light emission during the first emission duration, each respective pixel event corresponding to a breach of a respective comparator threshold indicative of a brightness level, each respective pixel event characterized by a respective electrical threshold value and a timestamp at which the respective electrical threshold value was breached.

Abstract: A mobile device includes a display, at least one sensor, and a wireless transceiver. The mobile device also includes control circuitry coupled to the display, the at least one sensor, and the wireless transceiver. The control circuitry is configured to obtain content primitives from the at least one sensor, to perform content provisioning operations to obtain content based at least in part on the content primitives, and to display the obtained content on the display, wherein at least some of the content is virtual content. In response to a bandwidth condition of the wireless communication channel being less than a threshold, the control circuitry is configured to perform adjusted content provisioning operations that involve increasing an amount of image processing operations performed by the mobile device to obtain the content.

Abstract: One implementation forms a composited stream of computer-generated reality (CGR) content using multiple data streams related to a CGR experience to facilitate recording or streaming. A media compositor obtains a first data stream of rendered frames and a second data stream of additional data. The rendered frame content (e.g., 3D models) represents real and virtual content rendered during a CGR experience at a plurality of instants in time. The additional data of the second data stream relates to the CGR experience, for example, relating to audio, audio sources, metadata identifying detected attributes of the CGR experience, image data, data from other devices involved in the CGR experience, etc. The media compositor forms a composited stream that aligns the rendered frame content with the additional data for the plurality of instants in time, for example, by forming time-stamped, n-dimensional datasets (e.g., images) corresponding to individual instants in time.

Abstract: A method and system for correcting luma data and chroma data from a composite signal for a plurality of lines are disclosed. The method and system include correlating line data of the composite signal from a portion of the plurality of lines to provide a plurality of correlation signals. The method and system also include generating a chroma suppression signal based on the plurality of correlation signals. The chroma suppression signal indicates whether to suppress a portion of the chroma data. The chroma suppression signal indicates that the portion of the chroma data should be suppressed when a large luma transition and the chroma data being beneath a threshold are detected.

Abstract: Processing signals in a video system may include detecting motion in at least one of a plurality of video frames, storing the results of the detecting to a buffer and comb filtering at least one future frame using the results stored in the buffer. Detecting the difference may include computing the difference between at least two of the plurality of frames on a pixel by pixel basis and comparing the difference to a threshold. The width and height of the buffer may be the same as the width and height of a video frame and a result may be stored in a location in the buffer corresponding to the location in the video frame where motion may have been detected. The buffer may hold the data representing motion from a plurality of frames. The processing may further include extracting chroma and luma information from a composite video signal by using a frame comb filter.

Abstract: Devices and methods are disclosed for decoding data in a data stream. One embodiment relates to a method of decoding data using an A/V decoder. In this embodiment, timing information is recovered from the input stream. The output rate of an output stream is adjusted using the recovered timing information, where the output stream has a clock that is asynchronous to a time reference of the input stream.

Abstract: A method and system for performing combing for PAL luma data is disclosed. The combing is performed for a display having a plurality of lines. The display is capable of depicting a frame including a horizontal boundary having a top edge and a bottom edge. A top line of the plurality of lines is at the top edge of the horizontal boundary. A bottom line of the plurality of lines is at the bottom edge of the horizontal boundary. The method and system includes providing a feedback multiplexer, a line delay and a feed forward multiplexer. The feedback multiplexer has a first input, a second input and a first output. The first input is for receiving luma data for a current line. The line delay has a delay input and a delay output, the delay input coupled with the first output. The delay output is coupled with the second input. The feed forward multiplexer has a third input, a fourth input and a second output. The third input is coupled with the delay output.

Abstract: A method and system for performing combing for PAL chroma data for a display having a plurality of lines is disclosed. The display is capable of depicting a frame including a horizontal boundary having a top edge and a bottom edge. The top line of the plurality of lines is at the top edge of the horizontal boundary, while a bottom line of the plurality of lines is at the bottom edge of the horizontal boundary. The method and system include replacing bottom line chroma data with previous line chroma data for the top line. The method and system also include replacing top line chroma data with subsequent line chroma data for the bottom line. The method and system can also provide three-line combing chroma data for a remaining portion of the plurality of lines.

Abstract: A method and system for correcting luma data and chroma data from a composite signal for a plurality of lines are disclosed. The method and system include correlating line data of the composite signal from a portion of the plurality of lines to provide a plurality of correlation signals. The method and system also include generating a chroma suppression signal based on the plurality of correlation signals. The chroma suppression signal indicates whether to suppress a portion of the chroma data. The chroma suppression signal indicates that the portion of the chroma data should be suppressed when a large luma transition and the chroma data being beneath a threshold are detected.

Abstract: A method and system for more efficiently loading a plurality of primitives for a scene into processors of a computer graphics system is disclosed. Each primitive has a top and a bottom. The primitives are ordered based on the top of each primitive. The system and method include providing at least one input, a merge circuit, a distributor, a feedback circuit and a controller. The input(s) is for receiving data relating to each primitive. The merge circuit is coupled with the input(s) and adds the data for a primitive having a top not lower than a current line. The distributor is coupled with the feedback circuit, eliminates an expired primitive and outputs the data for remaining primitives after the expired primitive has been removed. The expired primitive has a bottom above the current line. The feedback circuit is coupled to the merge circuit and the distributor and re-inputs to the merge circuit the data for the remaining primitives.

Abstract: A method and system for performing combing for PAL luma data is disclosed. The combing is performed for a display having a plurality of lines. The display is capable of depicting a frame including a horizontal boundary having a top edge and a bottom edge. A top line of the plurality of lines is at the top edge of the horizontal boundary. A bottom line of the plurality of lines is at the bottom edge of the horizontal boundary. The method and system includes providing a feedback multiplexer, a line delay and a feed forward multiplexer. The feedback multiplexer has a first input, a second input and a first output. The first input is for receiving luma data for a current line. The line delay has a delay input and a delay output, the delay input coupled with the first output. The delay output is coupled with the second input. The feed forward multiplexer has a third input, a fourth input and a second output. The third input is coupled with the delay output.

Abstract: A method and system for utilizing processor(s) and bypass processor(s) of a computer graphics system are disclosed. The processor(s) and bypass processor(s) render primitives, which are ordered based on their left corners. The method and system include providing a merge circuit, a distributor, a feedback circuit and a controller. The merge circuit determines left and right edges for each primitive. The distributor is coupled with feedback circuit and outputs a first portion of the primitives. The distributor provides a second portion of the primitives to the processor(s) and a third portion of the primitives to the bypass processor(s) if the first portion includes more primitives than there are processor(s). The second portion includes no more primitives than there are processor(s). The feedback circuit, coupled to the merge circuit, re-inputs a fourth portion of the primitives to the bypass processor(s) until the first portion has been rendered for a line.

Abstract: A method and system for performing combing for PAL luma data is disclosed. The combing is performed for a display having a plurality of lines. The display is capable of depicting a frame including a horizontal boundary having a top edge and a bottom edge. A top line of the plurality of lines is at the top edge of the horizontal boundary. A bottom line of the plurality of lines is at the bottom edge of the horizontal boundary. The method and system includes providing a feedback multiplexer, a line delay and a feed forward multiplexer. The feedback multiplexer has a first input, a second input and a first output. The first input is for receiving luma data for a current line. The line delay has a delay input and a delay output, the delay input coupled with the first output. The delay output is coupled with the second input. The feed forward multiplexer has a third input, a fourth input and a second output. The third input is coupled with the delay output.

Abstract: Devices and methods are disclosed for decoding data in a data stream. One embodiment relates to a method of decoding data using an A/V decoder. In this embodiment, timing information is recovered from the input stream. The output rate of an output stream is adjusted using the recovered timing information, where the output stream has a clock that is asynchronous to a time reference of the input stream.