Abstract: A bit-depth optimization engine reduces the hardware cost of a neural network. When training data is applied to a neural network during training routines, accuracy cost and hardware costs are generated. A hardware complexity cost generator generates costs for weights near bit-depth steps where the number of binary bits required to represent a weight decreases, such as from 2N to 2N?1, where one less binary bit is required. Gradients are generated from costs for each weight, and weights near bit-depth steps are easily selected since they have a large gradient, while weights far away from a bit-depth step have near-zero gradients. The selected weights are reduced during optimization. Over many cycles of optimization, a low-bit-depth neural network is generated that uses fewer binary bits per weight, resulting in lower hardware costs when the low-bit-depth neural network is manufactured on an Application-Specific Integrated Circuit (ASIC).

Abstract: A neural network accelerator reads encoded weights from memory. All 1 bits in a weight except the first three are discarded. The first three leading 1 bits in the weight are encoded as three bit-shift values to form the encoded weight. The three bit-shift values are applied to a bit shifter to shift a node input to obtain three shifted inputs that are accumulated to generate the node output. Node complexity is reduced since only 3 shifts are performed rather than up to 15 shifts for a 16-bit weight. The bit shifter and accumulator for a node can be implemented by Look-Up Tables (LUTs) without requiring a Multiply-Accumulate (MAC) cell in a Field-Programmable Gate Array (FPGA). Quantization bias is reduced using a histogram analyzer that determines a weighted average for each interval between quantized weights. The third bit-shift value is incremented for weights in the interval above the weighted average.

Abstract: A hybrid-resolution panoramic VR generator places High-Resolution (HR) patches from a ring of HR cameras onto a 360-degree Low-Resolution (LR) image from a LR camera pointing upward from the ring into a panoramic mirror that captures the combined field of view of all the multiple HR cameras, but at a lower resolution. Ghosting artifacts caused by parallax errors between adjacent HR cameras are eliminated because object placement is determined by the 360-degree LR image. Each HR image is homographicly projected into 3 projections by grouping objects of different depths to obtain homographic matrixes. The 360-degree LR image is upscaled to HR and a query patch is searched in search windows in the three projections for up to two adjacent HR images. Best-matching patches are weighted by similarity with the query patch and blended to generate a reconstructed patch placed at the query patch location in a reconstructed HR panorama image.

Abstract: A hybrid-resolution panoramic VR generator places High-Resolution (HR) patches from a ring of HR cameras onto a 360-degree Low-Resolution (LR) image from a LR camera pointing upward from the ring into a panoramic mirror that captures the combined field of view of all the multiple HR cameras, but at a lower resolution. Ghosting artifacts caused by parallax errors between adjacent HR cameras are eliminated because object placement is determined by the 360-degree LR image. Each HR image is homographicly projected into 3 projections by grouping objects of different depths to obtain homographic matrixes. The 360-degree LR image is upscaled to HR and a query patch is searched in search windows in the three projections for up to two adjacent HR images. Best-matching patches are weighted by similarity with the query patch and blended to generate a reconstructed patch placed at the query patch location in a reconstructed HR panorama image.

Abstract: A bit-depth optimization engine reduces the hardware cost of a neural network. When training data is applied to a neural network during training routines, accuracy cost and hardware costs are generated. A hardware complexity cost generator generates costs for weights near bit-depth steps where the number of binary bits required to represent a weight decreases, such as from 2N to 2N?1, where one less binary bit is required. Gradients are generated from costs for each weight, and weights near bit-depth steps are easily selected since they have a large gradient, while weights far away from a bit-depth step have near-zero gradients. The selected weights are reduced during optimization. Over many cycles of optimization, a low-bit-depth neural network is generated that uses fewer binary bits per weight, resulting in lower hardware costs when the low-bit-depth neural network is manufactured on an Application-Specific Integrated Circuit (ASIC).

Abstract: An image processor generates a Super-Resolution (SR) frame by upscaling. A Human Visual Preference Model (HVPM) helps detect random texture regions, where visual artifacts and errors are tolerated to allow for more image details, and immaculate regions having flat areas, corners, or regular structures, where details may be sacrificed to prevent annoying visual artifacts that seem to stand out more. A regularity or isotropic measurement is generated for each input pixel. More regular and less anisotropic regions are mapped as immaculate regions. Higher weights for blurring, smoothing, or blending from a single frame source are assigned for immaculate regions to reduce the likelihood of generated artifacts. In the random texture regions, multiple frames are used as sources for blending, and sharpening is increased to enhance details, but more artifacts are likely. These artifacts are more easily tolerated by humans in the random texture regions than in the regular-structure immaculate regions.

Abstract: An image processor generates a Super-Resolution (SR) frame by upscaling. A Human Visual Preference Model (HVPM) helps detect random texture regions, where visual artifacts and errors are tolerated to allow for more image details, and immaculate regions having flat areas, corners, or regular structures, where details may be sacrificed to prevent annoying visual artifacts that seem to stand out more. A regularity or isotropic measurement is generated for each input pixel. More regular and less anisotropic regions are mapped as immaculate regions. Higher weights for blurring, smoothing, or blending from a single frame source are assigned for immaculate regions to reduce the likelihood of generated artifacts. In the random texture regions, multiple frames are used as sources for blending, and sharpening is increased to enhance details, but more artifacts are likely. These artifacts are more easily tolerated by humans in the random texture regions than in the regular-structure immaculate regions.

Abstract: Embodiments of the present invention include apparatuses, systems and methods for multi-patch based super-resolution from a single video frame. Such embodiments include a scale-invariant self-similarity (SiSS) based super-resolution method. Instead of searching HR examples in a database or in LR image, the present embodiments may select the patches according to the SiSS characteristics of the patch itself, so that the computational complexity of the method may be reduced because there is not any search involved. To solve the problem of lack of relevant examples in natural images, the present embodiments may employ multi-shaped and multi-sized patches in HR image reconstruction. Additionally, embodiments may include steps for a hybrid weighing method for suppressing artifacts. Advantageously, certain embodiments of the method may be 10˜1,000 times faster than the example based SR approaches using patch searching and can achieve comparable HR image quality.

Abstract: Embodiments of the present invention include apparatuses, systems and methods for multi-patch based super-resolution from a single video frame. Such embodiments include a scale-invariant self-similarity (SiSS) based super-resolution method. Instead of searching HR examples in a database or in LR image, the present embodiments may select the patches according to the SiSS characteristics of the patch itself, so that the computational complexity of the method may be reduced because there is not any search involved. To solve the problem of lack of relevant examples in natural images, the present embodiments may employ multi-shaped and multi-sized patches in HR image reconstruction. Additionally, embodiments may include steps for a hybrid weighing method for suppressing artifacts. Advantageously, certain embodiments of the method may be 10˜1,000 times faster than the example based SR approaches using patch searching and can achieve comparable HR image quality.

Abstract: Various embodiments are directed to non-overlap region based automatic global alignment for ring camera image mosaic. The non-overlap region based homography calculation may be based on feature points of a planar target appearing in a non-overlap region among images captured by a multi-camera based video capture device. Other embodiments are described and claimed.

Abstract: Signal transition feature based coding for serial link is described herein. According to one embodiment, in response to a data stream transmitted onto a serial communication link, one or more bits of the data stream are encoded according to bit order determined based on a frequency of signal transitions of the data stream. As a result, a sequence of encoded data stream having a lower number of bit transitions with respect to the frequency of signal transitions of the data stream prior to the encoding is generated. Thereafter, the encoded data sequence is transmitted onto the serial communication link. Other methods and apparatuses are also described.

Abstract: Methods and apparatuses for error diffusion for display frame buffer power saving are described herein. According to one embodiment, pixels of a color plane of image data are stored in a first segment and a second segment of a frame buffer during a normal power state. During a low power state, an error diffusion operation is performed on the pixels to reduce a color depth of the pixels. Thereafter, at least a portion of the pixels with reduced color depth is stored in the first segment of the frame buffer during the low power state without accessing the second segment of the frame buffer. Other methods and apparatuses are also described.

Abstract: Signal transition feature based coding for serial link is described herein. According to one embodiment, in response to a data stream transmitted onto a serial communication link, one or more bits of the data stream are encoded according to bit order determined based on a frequency of signal transitions of the data stream. As a result, a sequence of encoded data stream having a lower number of bit transitions with respect to the frequency of signal transitions of the data stream prior to the encoding is generated. Thereafter, the encoded data sequence is transmitted onto the serial communication link. Other methods and apparatuses are also described.

Abstract: A speech recognition method includes several embodiments describing application of support vector machine analysis to a mouth region. Lip position can be accurately determined and used in conjunction with synchronous or asynchronous audio data to enhance speech recognition probabilities.

Abstract: Method and apparatus for an audiovisual continuous speech recognition (AVCSR) system using a coupled hidden Markov model (CHMM) are described herein. In one aspect, an exemplary process includes receiving an audio data stream and a video data stream, and performing continuous speech recognition based on the audio and video data streams using a plurality of hidden Markov models (HMMs), a node of each of the HMMs at a time slot being subject to one or more nodes of related HMMs at a preceding time slot. Other methods and apparatuses are also described.

Abstract: Various embodiments are directed to non-overlap region based automatic global alignment for ring camera image mosaic. The non-overlap region based homography calculation may be based on feature points of a planar target appearing in a non-overlap region among images captured by a multi-camera based video capture device. Other embodiments are described and claimed.

Abstract: A tree classifier may include a number of stages. Some stages may include monolithic classifiers, and other stages may be split into two or more classifiers.

Abstract: A phoneme and a viseme of a person may be modeled using a coupled hidden Markov model. The coupled hidden Markov model and a second model may be compared to identify the person.

Abstract: A tree classifier may include a number of stages. Some stages may include monolithic classifiers, and other stages may be split into two or more classifiers.

Abstract: Method and apparatus for an audiovisual continuous speech recognition (AVCSR) system using a coupled hidden Markov model (CHMM) are described herein. In one aspect, an exemplary process includes receiving an audio data stream and a video data stream, and performing continuous speech recognition based on the audio and video data streams using a plurality of hidden Markov models (HMMs), a node of each of the HMMs at a time slot being subject to one or more nodes of related HMMs at a preceding time slot. Other methods and apparatuses are also described.