Abstract: A method for analyzing images to generate a plurality of output features includes receiving input features of the image and performing Fourier transforms on each input feature. Kernels having coefficients of a plurality of trained features are received and on-the-fly Fourier transforms (OTF-FTs) are performed on the coefficients in the kernels. The output of each Fourier transform and each OTF-FT are multiplied together to generate a plurality of products and each of the products are added to produce one sum for each output feature. Two-dimensional inverse Fourier transforms are performed on each sum.

Abstract: This invention predicts that intra mode prediction is more effective for the macroblocks where motion estimation in inter mode prediction fails. This failure is indicated by a large value of the inter mode SAD. This invention performs intra mode prediction for only macro blocks have larger inter mode SADs. The definition of a large inter mode SAD differs for different content. This invention compares the inter mode SAD of a current macroblock with an adaptive threshold. This adaptive threshold depends on the average and variance of the SADs of the previous predicted frame. An adaptive threshold is calculated for each new predictive frame.

Abstract: A method of rate control in coding of a video sequence to generate a compressed bit stream is provided that includes computing a sequence base quantization step size for a sequence of pictures in the video sequence, computing a picture base quantization step size for a picture in the sequence of pictures based on the sequence base quantization step size, a type of the picture, and a level of the picture in a rate control hierarchy, and coding the picture using the picture base quantization step size to generate a portion of the compressed bit stream.

Abstract: Advanced driver assistance systems can be designed to recognize and to classify traffic signs under real time constraints, and under a wide variety of visual conditions. This disclosure provides techniques that employ binary masks extracted by color space segmentation, with a different binary mask generated for each sign shape. Temporal tracking is employed to add robustness to the detection system. The system is generic, and is trainable to the traffic signs used in various countries.

Abstract: Techniques including receiving a distorted image from a camera disposed about a vehicle, detecting, in the distorted image, corner points associated with a target object, mapping the corner points to a distortion corrected domain based on one or more camera parameters, mapping the corner points and lines between the corner points back to a distorted domain based on the camera parameters, interpolating one or more intermediate points to generate lines between the corner points in the distortion corrected domain mapping the corner points and the lines between the corner points back to a distorted domain based on the camera parameters, and adjusting a direction of travel of the vehicle based on the located target object.

Abstract: In described examples, an integrated circuit includes a memory storing weights and biases, an N-bit fixed point matrix operations accelerator, and a processor. Starting with a first convolution layer, a convolution layer modeled using the processor receives input feature values. A feature scale and weight scale are reduced if an accumulator scale is greater than a maximum bias scale. The input feature values are rescaled using the feature scale, the weights are quantized using the weight scale, and the biases are quantized using the feature scale and weight scale. The rescaled input feature values and quantized weights and biases are convolved using the N-bit fixed point matrix operations accelerator to generate output feature values. The process repeats from the receive action using the output feature values as the input feature values of the next convolution layer. The process then repeats for all layers, feeding back an output feature range.

Abstract: An image processing system includes a processor and optical flow (OF) determination logic for quantifying relative motion of a feature present in a first frame of video and a second frame of video that provide at least one of temporally and spatially ordered images with respect to the two frames of video. The OF determination logic configures the processor to implement performing OF estimation between the first frame and second frame using a pyramidal block matching (PBM) method to generate an initial optical flow (OF) estimate at a base pyramid level having integer pixel resolution, and refining the initial OF estimate using at least one pass of a modified Lucas-Kanade (LK) method to provide a revised OF estimate having fractional pixel resolution.

Abstract: Described examples include an integrated circuit including a vector multiply unit including a plurality of multiply/accumulate nodes, in which the vector multiply unit is operable to provide an output from the multiply/accumulate nodes, a first data feeder operable to provide first data to the vector multiply unit in vector format, and a second data feeder operable to provide second data to the vector multiply unit in vector format.

Abstract: Techniques including receiving a distorted image from a camera disposed about a vehicle, detecting, in the distorted image, corner points associated with a target object, mapping the corner points to a distortion corrected domain based on one or more camera parameters, mapping the corner points and lines between the corner points back to a distorted domain based on the camera parameters, interpolating one or more intermediate points to generate lines between the corner points in the distortion corrected domain mapping the corner points and the lines between the corner points back to a distorted domain based on the camera parameters, and adjusting a direction of travel of the vehicle based on the located target object.

Abstract: A method of rate control in coding of a video sequence to generate a compressed bit stream is provided that includes computing a sequence base quantization step size for a sequence of pictures in the video sequence, computing a picture base quantization step size for a picture in the sequence of pictures based on the sequence base quantization step size, a type of the picture, and a level of the picture in a rate control hierarchy, and coding the picture using the picture base quantization step size to generate a portion of the compressed bit stream.

Abstract: A method for estimating time to collision (TTC) of a detected object in a computer vision system is provided that includes determining a three dimensional (3D) position of a camera in the computer vision system, determining a 3D position of the detected object based on a 2D position of the detected object in an image captured by the camera and an estimated ground plane corresponding to the image, computing a relative 3D position of the camera, a velocity of the relative 3D position, and an acceleration of the relative 3D position based on the 3D position of the camera and the 3D position of the detected object, wherein the relative 3D position of the camera is relative to the 3D position of the detected object, and computing the TTC of the detected object based on the relative 3D position, the velocity, and the acceleration.

Abstract: A method for convolution in a convolutional neural network (CNN) is provided that includes accessing a coefficient value of a filter corresponding to an input feature map of a convolution layer of the CNN, and performing a block multiply accumulation operation on a block of data elements of the input feature map, the block of data elements corresponding to the coefficient value, wherein, for each data element of the block of data elements, a value of the data element is multiplied by the coefficient value and a result of the multiply is added to a corresponding data element in a corresponding output block of data elements comprised in an output feature map.

Abstract: In described examples of a method for quantizing data for a convolutional neural network (CNN) is provided. A set of data is received and quantized the using a power-of-2 parametric activation (PACT2) function. The PACT2 function arranges the set of data as a histogram and discards a portion of the data corresponding to a tail of the histogram to form a remaining set of data. A clipping value is determined by expanding the remaining set of data to a nearest power of two value. The set of data is then quantized using the clipping value. With PACT2, a model can be quantized either using post training quantization or using quantization aware training. PACT2 helps a quantized model to achieve close accuracy compared to the corresponding floating-point model.

Abstract: A method for convolution in a convolutional neural network (CNN) is provided that includes accessing a coefficient value of a filter corresponding to an input feature map of a convolution layer of the CNN, and performing a block multiply accumulation operation on a block of data elements of the input feature map, the block of data elements corresponding to the coefficient value, wherein, for each data element of the block of data elements, a value of the data element is multiplied by the coefficient value and a result of the multiply is added to a corresponding data element in a corresponding output block of data elements comprised in an output feature map.

Abstract: A method for dynamically quantizing feature maps of a received image. The method includes convolving an image based on a predicted maximum value, a predicted minimum value, trained kernel weights and the image data. The input data is quantized based on the predicted minimum value and predicted maximum value. The output of the convolution is computed into an accumulator and re-quantized. The re-quantized value is output to an external memory. The predicted min value and the predicted max value are computed based on the previous max values and min values with a weighted average or a pre-determined formula. Initial min value and max value are computed based on known quantization methods and utilized for initializing the predicted min value and predicted max value in the quantization process.

Abstract: A method for estimating time to collision (TTC) of a detected object in a computer vision system is provided that includes determining a three dimensional (3D) position of a camera in the computer vision system, determining a 3D position of the detected object based on a 2D position of the detected object in an image captured by the camera and an estimated ground plane corresponding to the image, computing a relative 3D position of the camera, a velocity of the relative 3D position, and an acceleration of the relative 3D position based on the 3D position of the camera and the 3D position of the detected object, wherein the relative 3D position of the camera is relative to the 3D position of the detected object, and computing the TTC of the detected object based on the relative 3D position, the velocity, and the acceleration.

Abstract: Estimation of the ground plane of a three dimensional (3D) point cloud based modifications to the random sample consensus (RANSAC) algorithm is provided. The modifications may include applying roll and pitch constraints to the selection of random planes in the 3D point cloud, using a cost function based on the number of inliers in the random plane and the number of 3D points below the random plane in the 3D point cloud, and computing a distance threshold for the 3D point cloud that is used in determining whether or not a 3D point in the 3D point cloud is an inlier of a random plane.

Abstract: A method for video object detection includes detecting an object in a first video frame, and selecting a first interest point and a second interest point of the object. The first interest point is in a first region of interest located at a first corner of a box surrounding the object. The second interest point is in a second region of interest located at a second corner of the box. The second corner is diagonally opposite the first corner. A first optical flow of the first interest point and a second optical flow of the second interest point are determined. A location of the object in a second video frame is estimated by determining, in the second video frame, a location of the first interest point based on the first optical flow and a location of the second interest point based on the second optical flow.

Abstract: A method of rate control in coding of a video sequence to generate a compressed bit stream is provided that includes computing a sequence base quantization step size for a sequence of pictures in the video sequence, computing a picture base quantization step size for a picture in the sequence of pictures based on the sequence base quantization step size, a type of the picture, and a level of the picture in a rate control hierarchy, and coding the picture using the picture base quantization step size to generate a portion of the compressed bit stream.

Abstract: Estimation of the ground plane of a three dimensional (3D) point cloud based modifications to the random sample consensus (RANSAC) algorithm is provided. The modifications may include applying roll and pitch constraints to the selection of random planes in the 3D point cloud, using a cost function based on the number of inliers in the random plane and the number of 3D points below the random plane in the 3D point cloud, and computing a distance threshold for the 3D point cloud that is used in determining whether or not a 3D point in the 3D point cloud is an inlier of a random plane.