Abstract: A system receives a neural network model that includes asymmetric operations. Each asymmetric operation includes one or more fixed-point operands that are asymmetrically-quantized from corresponding floating-point operands. The system compiles a given asymmetric operation of the neural network model into a symmetric operation that includes a combined bias value. A compiler computes the combined bias value is a constant by merging at least zero points of input and output of the given asymmetric operation. The system then generates a symmetric neural network model including the symmetric operation for inference hardware to execute in fixed-point arithmetic.

Abstract: A video processing circuit includes an input buffer, an online adaptation circuit, and an artificial intelligence (AI) super-resolution (SR) circuit. The input buffer receives input low-resolution (LR) frames and high-resolution (HR) frames from a video source over a network. The online adaptation circuit forms training pairs, and calculates an update to representative features that characterize the input LR frames using the training pairs. Each training pair formed by one of the input LR frames and one of the HR frames. The AI SR circuit receives the input LR frames from the input buffer and the representative features from the online adaptation circuit. Concurrently with calculating the update to the representative features, the AI SR circuit generates SR frames for display from the input LR frames based on the representative features. Each SR frame has a higher resolution than a corresponding one of the input LR frames.

Abstract: A device produces a dolly zoom effect with automatic focal length adjustment. The device uses a camera to capture an initial image including at least a foreground object and a background. The device includes a size tracking circuit to identify the size of the foreground object in the initial image. The device further includes a focal length control circuit. The focal length control circuit calculates an adjusted focal length of the camera to maintain the size of the foreground object in subsequently captured images.

Abstract: A frame interpolation method for generating a third image frame interpolated between a first image frame and a second image frame includes: performing motion estimation upon a first input image frame and a second input image frame, to obtain a single-directional motion, wherein the first input image frame is derived from the first image frame, and the second input image frame is derived from the second image frame; scaling the single-directional motion according to a time point of the third image frame, to generate a scaled motion; deriving a forward-warped result from a result of performing a forward warping operation and a first inverse operation upon the scaled motion; performing a second inverse operation upon the forward-warped result, to generate an inversed result; and generating the third image frame according to the first image frame, the second image frame, the forward-warped result, and the inversed result.

Abstract: A method of feature extraction from an image can include receiving the image including pixels, generating confidence values corresponding to positions of the pixels in the image by an artificial intelligence (AI) based feature extractor, selecting a first position among the positions of the pixels in the image, a first confidence value among the generated confidence values at the first position being higher than a first threshold, and generating a final set of keypoint-descriptor pairs based on the confidence values corresponding to positions of the pixels in the image. The final set of keypoint-descriptor pairs includes at least two keypoint-descriptor pairs corresponding to the first position among the positions of the pixels in the image.

Abstract: Aspects of the disclosure provide a frame processor for processing frames with aliasing artifacts. For example, the frame processor can include a super-resolution (SR) and anti-aliasing (AA) engine and an attention reference frame generator coupled to the SR and AA engine. The SR and AA engine can be configured to enhance resolution and remove aliasing artifacts of a frame to generate a first high-resolution frame with aliasing artifacts and a second high-resolution frame with aliasing artifacts removed. The attention reference frame generator can be configured to generate an attention reference frame based on the first high-resolution frame and the second high-resolution frame.

Abstract: A booster engine enhances the quality of a frame sequence. The booster engine receives, from a first stage circuit, the frame sequence with quality degradation in at least a frame. The the quality degradation includes at least one of uneven resolution and uneven frame per second (FPS). The booster engine queries an information repository for reference information on the frame, using a query input based on at least a region of the frame to obtain a query output. The booster engine then applies a neural network to the query input and the query output to generate an optimized frame, and sends an enhanced frame sequence including the optimized frame to a second stage circuit.

Abstract: The quality of a frame sequence is enhanced by a booster engine collaborating with a first stage circuit. The first stage circuit adjusts the quality degradation of the frame sequence when a condition in constrained resources is detected. The quality degradation includes at least one of uneven resolution and uneven frame per second (FPS). The booster engine receives the frame sequence from the first stage circuit, and generates an enhanced frame sequence based on the frame sequence for transmission to a second stage circuit.

Abstract: A system collects a training dataset for training an artificial intelligence (AI) model. The system receives high-resolution (HR) images and information of one or more regions-of-interest (ROIs) in the HR images. The system maps a stride distribution to the ROIs, and samples the HR images with non-uniform strides according to the ROIs and the stride distribution to generate corresponding low-resolution (LR) images. The system then trains the AI model to perform super-resolution (SR) operations using training pairs formed by the HR images and respective corresponding LR images.

Abstract: Aspects of the disclosure provide a device for processing frames with aliasing artifacts. For example, the device can include a motion estimation circuit, a warping circuit coupled to the motion estimation circuit, and a temporal decision circuit coupled to the warping circuit. The motion estimation circuit can estimate a motion value between a current frame and a previous frame. The warping circuit can warp the previous frame based on the motion value such that the warped previous frame is aligned with the current frame and determine whether the current frame and the warped previous frame are consistent. The temporal decision circuit can generate an output frame, the output frame including either the current frame and the warped previous frame when the current frame and the warped previous frame are consistent, or the current frame when the current frame and the warped previous frame are not consistent.

Abstract: A system produces a dolly zoom effect by utilizing side view information. The system first captures a main image at a main location. The main image includes at least a foreground object of a given size and a background. The system calculates one or more side view locations based on a zoom-in factor to be applied to the background and an estimated size of the foreground object. The system then guides a user to capture one or more side view images at the one or more side view locations. The foreground object of the given size is superimposed onto a zoomed-in background. Then the side view information is used by the system to perform image inpainting.

Abstract: A video processing system includes an input port and a video processing circuit. The input port obtains device information of a display panel. The video processing circuit obtains an input frame and the device information, configures an image enhancement operation according to the device information, generates an output frame by performing the image enhancement operation upon the input frame, and transmits the output frame to the display panel for video playback.

Abstract: An analog circuit is calibrated to perform neural network computing. Calibration input is provided to a pre-trained neural network that includes at least a given layer having pre-trained weights stored in the analog circuit. The analog circuit performs tensor operations of the given layer using the pre-trained weights. Statistics of calibration output from the analog circuit is calculated. Normalization operations to be performed during neural network inference are determined. The normalization operations incorporate the statistics of the calibration output and are performed at a normalization layer that follows the given layer. A configuration of the normalization operations is written into memory while the pre-trained weights stay unchanged.

Abstract: A computing device operative to perform parallel computations. The computing device includes a controller unit to assign workgroups to a set of batches. Each batch includes a program counter shared by M workgroups assigned to the batch, where M is a positive integer determined according to a configurable batch setting. Each batch further includes a set of thread processing units operative to execute, in parallel, a subset of work items in each of the M workgroups. Each batch further includes a spilling memory to store intermediate data of the M workgroups when one or more workgroups in the M workgroups encounters a synchronization barrier.

Abstract: An image processing circuit stores a training database and models in memory. The image processing circuit includes an attribute identification engine to identify an attribute from an input image according to a model stored in the memory. By enhancing the input image based on the identified attribute, a picture quality (PQ) engine in the image processing circuit generates an output image for display. The image processing circuit further includes a data collection module to generate a labeled image based on the input image labeled with the identified attribute, and to add the labeled image to the training database. A training engine in the image processing circuit then re-trains the model using the training database.

Abstract: A video processing circuit includes an input buffer, an online adaptation circuit, and an artificial intelligence (AI) super-resolution (SR) circuit. The input buffer receives input low-resolution (LR) frames and high-resolution (HR) frames from a video source over a network. The online adaptation circuit forms training pairs, and calculates an update to representative features that characterize the input LR frames using the training pairs. Each training pair formed by one of the input LR frames and one of the HR frames. The AI SR circuit receives the input LR frames from the input buffer and the representative features from the online adaptation circuit. Concurrently with calculating the update to the representative features, the AI SR circuit generates SR frames for display from the input LR frames based on the representative features. Each SR frame has a higher resolution than a corresponding one of the input LR frames.

Abstract: An image processing circuit performs super-resolution (SR) operations. The image processing circuit includes memory to store multiple parameter sets of multiple artificial intelligent (AI) models. The image processing circuit further includes an image guidance module, a parameter decision module, and an SR engine. The image guidance module operates to detect a representative feature in an image sequence including a current frame and past frames within a time window. The parameter decision module operates to adjust parameters of one or more AI models based on a measurement of the representative feature. The SR engine operates to process the current frame using the one or more AI models with the adjusted parameters to thereby generate a high-resolution image for display.

Abstract: An image processing apparatus includes a super-resolution (SR) circuit and a resizer circuit. The SR circuit performs an SR operation upon a first image to generate a second image, wherein a resolution of the second image is not lower than a resolution of the first image, and the SR operation is based, at least in part, on one or more artificial intelligence (AI) models. The resizer circuit performs a resize operation upon the second image to generate a third image, wherein a resolution of the third image is not lower than the resolution of the second image, and no AI model is involved in the resize operation.

Abstract: A method of feature extraction from an image can include receiving the image including pixels, generating confidence values corresponding to positions of the pixels in the image by an artificial intelligence (AI) based feature extractor, selecting a first position among the positions of the pixels in the image, a first confidence value among the generated confidence values at the first position being higher than a first threshold, and generating a final set of keypoint-descriptor pairs based on the confidence values corresponding to positions of the pixels in the image. The final set of keypoint-descriptor pairs includes at least two keypoint-descriptor pairs corresponding to the first position among the positions of the pixels in the image.

Abstract: Aspects of the disclosure provide a device for processing frames with aliasing artifacts. For example, the device can include a motion estimation circuit, a warping circuit coupled to the motion estimation circuit, and a temporal decision circuit coupled to the warping circuit. The motion estimation circuit can estimate a motion value between a current frame and a previous frame. The warping circuit can warp the previous frame based on the motion value such that the warped previous frame is aligned with the current frame and determine whether the current frame and the warped previous frame are consistent. The temporal decision circuit can generate an output frame, the output frame including either the current frame and the warped previous frame when the current frame and the warped previous frame are consistent, or the current frame when the current frame and the warped previous frame are not consistent.