Abstract: Video quality analysis may be used in many multimedia transmission and communication applications, such as encoder optimization, stream selection, and/or video reconstruction. An objective VQA metric that accurately reflects the quality of processed video relative to a source unprocessed video may take into account both spatial measures and temporal, motion-based measures when evaluating the processed video. Temporal measures may include differential motion metrics indicating a difference between a frame difference of a plurality of frames of the processed video relative to that of a corresponding plurality of frames of the source video. In addition, neural networks and deep learning techniques can be used to develop additional improved VQA metrics that take into account both spatial and temporal aspects of the processed and unprocessed videos.

Abstract: A disclosed configuration includes a system (or a computer implemented method or a non-transitory computer readable medium) for automatically preprocessing higher dynamic range image data into lower dynamic range image data through a data adaptive tuning process. By automatically preprocessing the higher dynamic range image data into the lower dynamic range image data through the data adaptive tuning process, an existing encoding process for encoding the standard dynamic range image data can be applied to the lower dynamic range image data while preserving metadata sufficient to recover image fidelity even in the high dynamic range. In one aspect, the system (or a computer implemented method or a non-transitory computer readable medium) provides for backwards compatibility between high dynamic range video services and existing standard dynamic range services. In one aspect, regrading is applied in a domain that is perceptually more uniform than the domain it is initially presented.

Abstract: A video coding system in which video images of a video bitstream are rescaled prior to encoding, and again at the decoder upon reception. When encoding a given video frame, the video encoder deduces a level of resampling to apply to a reference frame in order to properly predict blocks in the given video frame or the full given video frame, and carries out one or more predictions by first applying a resampling process on the reference frame data at the deduced level. To decode the given video frame of the bitstream, a video decoder compares a size of the given video frame to sizes of a reference frame to determine a level of resampling for the reference frame data, and carries out predictions to generate predicted data by first applying the determined level of resampling to the reference frame data.

Abstract: Video quality analysis may be used in many multimedia transmission and communication applications, such as encoder optimization, stream selection, and/or video reconstruction. An objective VQA metric that accurately reflects the quality of processed video relative to a source unprocessed video may take into account both spatial measures and temporal, motion-based measures when evaluating the processed video. Temporal measures may include differential motion metrics indicating a difference between a frame difference of a plurality of frames of the processed video relative to that of a corresponding plurality of frames of the source video. In addition, neural networks and deep learning techniques can be used to develop additional improved VQA metrics that take into account both spatial and temporal aspects of the processed and unprocessed videos.

Abstract: A disclosed configuration includes a system (or a computer implemented method or a non-transitory computer readable medium) for automatically preprocessing higher dynamic range image data into lower dynamic range image data through a data adaptive tuning process. By automatically preprocessing the higher dynamic range image data into the lower dynamic range image data through the data adaptive tuning process, an existing encoding process for encoding the standard dynamic range image data can be applied to the lower dynamic range image data while preserving metadata sufficient to recover image fidelity even in the high dynamic range. In one aspect, the system (or a computer implemented method or a non-transitory computer readable medium) provides for backwards compatibility between high dynamic range video services and existing standard dynamic range services. In one aspect, regrading is applied in a domain that is perceptually more uniform than the domain it is initially presented.

Abstract: Encoding and decoding systems disclosed enhance the AV1/VPX codecs in the context of 8-bit SDR video and 10-bit HDR video content, for applications including streaming and high quality coding for content contribution editing. For SDR content, lapped biorthogonal transforms for near lossless applications and used and optimized resampling filter pairs for adaptive resolution coding in streaming applications are used. For HDR content, a data adaptive grading technique in conjunction with the VP9/VP10 encoder may be used. The encoding/decoding system provides substantial value in the coding of HDR content, and provides backward compatibility with SDR.

Abstract: Methods and systems use a video sensor grid over an area, and extensive signal processing, to create a model-based view of reality. Grid-based synchronous capture, point cloud generation and refinement, morphology, polygonal tiling and surface representation, texture mapping, data compression, and system-level components for user-directed signal processing, is used to create, at user demand, a virtualized world, viewable from any location in an area, in any direction of gaze, at any time within an interval of capture. This data stream is transmitted for near-term network-based delivery, and 5G. Finally, that virtualized world, because it is inherently model-based, is integrated with augmentations (or deletions), creating a harmonized and photorealistic mix of real, and synthetic, worlds. This provides a fully immersive, mixed reality world, in which full interactivity, using gestures, is supported.

Abstract: Artificial intelligence based techniques are used for analysis of 3D objects in conjunction with each other. A 3D model of two or more 3D objects is generated. Features of 3D objects are matched to develop a correspondence between the 3D objects. Two 3D objects are geometrically mapped and an object is overlayed on another 3D object to obtain a superimposed object. Match analysis of 3D objects is performed based on machine learning based models to determine how well the objects are spatially matched. The analysis of the objects is used in augmented reality applications.

Abstract: Embodiments of the invention recognize human visual gestures, as captured by image and video sensors, to develop a visual language for a variety of human computer interfaces. One embodiment provides a method for recognizing a hand gesture positioned by a user hand. The method includes steps of capturing a digital color image of a user hand against a background, applying a general parametric model to the digital color image of the user hand to generate a specific parametric template of the user hand, receiving a second digital image of the user hand positioned to represent a hand gesture, detecting a hand contour of the hand gesture based at least in part on the specific parametric template of the user hand, and recognizing the hand gesture based at least in part on the detected hand contour. Other embodiments include recognizing hand gestures, facial gestures or body gestures captured in a video.

Abstract: Embodiments of the invention receive videos and feedback data associated with the videos from a client device and adaptively encode the videos based on the feedback data. The encoded videos are suitable to be transmitted over a network and displayed on the client device. Embodiments of an encoding server adaptively changes resolution of a video on the fly or scale the video quality up or down based on the factors described by the feedback data, including network condition for transmitting the encoded video, network delay, encoder and decoder processing capacity and feedback from viewers of the decoded video. Furthermore, the encoding server adaptively encodes the video based on a combination of various factors described by the feedback data.

Abstract: A solution is provided to estimate motion vectors of a video. A multistage motion vector prediction engine is configured to estimate multiple best block-matching motion vectors for each block in each video frame of the video. For each stage of the motion vector estimation for a block of a video frame, the prediction engine selects a test vector form a predictor set of test vectors, computes a rate-distortion optimization (RDO) based metric for the selected test vector, and selects a subset of test vectors as individual best matched motion vectors based on the RDO based metric. The selected individual best matched motion vectors are compared and a total best matched motion vector is selected based on the comparison. The prediction engine selects iteratively applies one or more global matching criteria to the selected best matched motion vector to select a best matched motion vector for the block of pixels.

Abstract: Embodiments of the invention recognize human visual gestures, as captured by image and video sensors, to develop a visual language for a variety of human computer interfaces. One embodiment of the invention provides a computer-implement method for recognizing a visual gesture portrayed by a part of human body such as a human hand, face or body. The method includes steps of receiving the visual signature captured in a video having multiple video frames, determining a gesture recognition type from multiple gesture recognition types including shaped-based gesture, position-based gesture, motion-assisted and mixed gesture that combining two different gesture types. The method further includes steps of selecting a visual gesture recognition process based on the determined gesture type and applying the selected visual gesture recognition process to the multiple video frames capturing the visual gesture to recognize the visual gesture.

Abstract: Embodiments of the invention recognize human visual gestures, as captured by image and video sensors, to develop a visual language for a variety of human computer interfaces. One embodiment of the invention provides a computer-implement method for recognizing a visual gesture portrayed by a part of human body such as a human hand, face or body. The method includes steps of receiving the visual signature captured in a video having multiple video frames, determining a gesture recognition type from multiple gesture recognition types including shaped-based gesture, position-based gesture, motion-assisted and mixed gesture that combining two different gesture types. The method further includes steps of selecting a visual gesture recognition process based on the determined gesture type and applying the selected visual gesture recognition process to the multiple video frames capturing the visual gesture to recognize the visual gesture.

Abstract: A solution is provided to estimate motion vectors of a video. A multistage motion vector prediction engine is configured to estimate multiple best block-matching motion vectors for each block in each video frame of the video. For each stage of the motion vector estimation for a block of a video frame, the prediction engine selects a test vector form a predictor set of test vectors, computes a rate-distortion optimization (RDO) based metric for the selected test vector, and selects a subset of test vectors as individual best matched motion vectors based on the RDO based metric. The selected individual best matched motion vectors are compared and a total best matched motion vector is selected based on the comparison. The prediction engine selects iteratively applies one or more global matching criteria to the selected best matched motion vector to select a best matched motion vector for the block of pixels.

Abstract: Embodiments of the invention receive videos and feedback data associated with the videos from a client device and adaptively encode the videos based on the feedback data. The encoded videos are suitable to be transmitted over a network and displayed on the client device. Embodiments of an encoding server adaptively changes resolution of a video on the fly or scale the video quality up or down based on the factors described by the feedback data, including network condition for transmitting the encoded video, network delay, encoder and decoder processing capacity and feedback from viewers of the decoded video. Furthermore, the encoding server adaptively encodes the video based on a combination of various factors described by the feedback data.

Abstract: Embodiments of the invention recognize human visual gestures, as captured by image and video sensors, to develop a visual language for a variety of human computer interfaces. One embodiment provides a method for recognizing a hand gesture positioned by a user hand. The method includes steps of capturing a digital color image of a user hand against a background, applying a general parametric model to the digital color image of the user hand to generate a specific parametric template of the user hand, receiving a second digital image of the user hand positioned to represent a hand gesture, detecting a hand contour of the hand gesture based at least in part on the specific parametric template of the user hand, and recognizing the hand gesture based at least in part on the detected hand contour. Other embodiments include recognizing hand gestures, facial gestures or body gestures captured in a video.

Abstract: Motion estimation is the science of predicting the current frame in a video sequence from the past frame (or frames), by slicing it into rectangular blocks of pixels, and matching these to past such blocks. The displacement in the spatial position of the block in the current frame with respect to the past frame is called the motion vector. This method of temporally decorrelating the video sequence by finding the best matching blocks from past reference frames—motion estimation—makes up about 80% or more of the computation in a video encoder. In this patent disclosure, we define a method of searching only a very sparse subset of possible displacement positions (or motion vectors) among all possible ones, to see if we can get a good enough match, and terminate early. This sparse subset of motion vectors is preselected using prior knowledge and extensive testing on video sequences.

Abstract: Embodiments of the invention recognize human visual gestures, as captured by image and video sensors, to develop a visual language for a variety of human computer interfaces. One embodiment of the invention provides a computer-implement method for recognizing a visual gesture portrayed by a part of human body such as a human hand, face or body. The method includes steps of receiving the visual signature captured in a video having multiple video frames, determining a gesture recognition type from multiple gesture recognition types including shaped-based gesture, position-based gesture, motion-assisted and mixed gesture that combining two different gesture types. The method further includes steps of selecting a visual gesture recognition process based on the determined gesture type and applying the selected visual gesture recognition process to the multiple video frames capturing the visual gesture to recognize the visual gesture.

Abstract: Embodiments of the invention receive videos and feedback data associated with the videos from a client device and adaptively encode the videos based on the feedback data. The encoded videos are suitable to be transmitted over a network and displayed on the client device. Embodiments of an encoding server adaptively changes resolution of a video on the fly or scale the video quality up or down based on the factors described by the feedback data, including network condition for transmitting the encoded video, network delay, encoder and decoder processing capacity and feedback from viewers of the decoded video. Furthermore, the encoding server adaptively encodes the video based on a combination of various factors described by the feedback data.

Abstract: Disclosed are methods and apparatus for composing and communicating Digital Composition Coded Multisensory Messages (DCC MSMs). The present invention also relates to the design, composition, creation, and communication of the multisensory messages. Multisensory messages, while rich in content and meaning, are to be composable from a great variety of platforms, from cell phones to mainframes.