Abstract: Embodiments are disclosed for a single RGBIR camera module that is capable of imaging at both the visible and IR wavelengths. In some embodiments, a camera module comprises: an image sensor comprising: a microlens array; a color filter array (CFA) comprising a red filter, a blue filter, a green filter and at least one infrared (IR) filter; and a pixel array comprising pixels to convert light received through the color filter array into electrical signals; and an image signal processor (ISP) configured to: initiate capture of a first frame by reading signal pixels from the pixel array; initiate capture of a second frame by reading IR pixels from the pixel array; align the first and second frames; and extract the second frame from the first frame to generate a third enhanced frame.

Abstract: An apparatus to facilitate partially-frozen neural networks for efficient computer vision systems is disclosed. The apparatus includes a frozen core to store fixed weights of a machine learning model, one or more trainable cores coupled to the frozen core, the one or more trainable cores comprising multipliers for trainable weights of the machine learning model, and wherein the alpha blending layer includes a trainable alpha blending parameter, and wherein the trainable alpha blending parameter is a function of a trainable parameter, a sigmoid function, and outputs of frozen and trainable blocks in a preceding layer of the machine learning model.

Abstract: An apparatus to facilitate partially-frozen neural networks for efficient computer vision systems is disclosed. The apparatus includes a frozen core to store fixed weights of a machine learning model, one or more trainable cores coupled to the frozen core, the one or more trainable cores comprising multipliers for trainable weights of the machine learning model, and wherein the alpha blending layer includes a trainable alpha blending parameter, and wherein the trainable alpha blending parameter is a function of a trainable parameter, a sigmoid function, and outputs of frozen and trainable blocks in a preceding layer of the machine learning model.

Abstract: An example apparatus for adjusting eye gaze in images one or more processors to execute instructions to bidirectionally train a neural network; access a target angle and an input image, the input image including an eye in a first position; generate a vector field with the neural network; and generate a gaze-adjusted image based on the vector field, the gaze-adjusted image including the eye in a second position.

Abstract: An example apparatus for adjusting eye gaze in images one or more processors to execute instructions to bidirectionally train a neural network; access a target angle and an input image, the input image including an eye in a first position; generate a vector field with the neural network; and generate a gaze-adjusted image based on the vector field, the gaze-adjusted image including the eye in a second position.

Abstract: An apparatus, method, and a computer readable medium for attenuating visual artifacts in processed images. An annotated dataset of images to be processed by an image processing system is created. An adversarial control network is trained to operate as an image quality expert in classifying images. After the adversarial control network has been trained, the adversarial control network is used to supervise the image processing system on-the-fly.

Abstract: An image capture device is described. The image capture device includes an array of pixels, each pixel including a photodetector. A Bayer pattern color filter is disposed over a 4×4 subset of pixels in the array of pixels. The Bayer pattern color filter defines a first 2×2 subset of pixels sensitive to red light; a second 2×2 subset of pixels sensitive to green light; a third 2×2 subset of pixels sensitive to green light; and a fourth 2×2 subset of pixels sensitive to blue light. A set of 1×1 on-chip lenses (OCLs) includes a different 1×1 OCL disposed over each pixel in the second 2×2 subset of pixels and the third 2×2 subset of pixels. A set of 2×1 OCLs or 2×2 OCLs includes a 2×1 OCL or a 2×2 OCL disposed over each pixel in the first 2×2 subset of pixels and the fourth 2×2 subset of pixels.

Abstract: A recommendation system for recommending a target feature value for a target feature for a target deployment is provided. The recommendation system, for each of a plurality of deployments, collects feature values for the features of that deployment. The recommendation system then generates a model for recommending a target feature value for the target feature based on the collected feature values of the features for the deployments. The recommendation system applies the model to the features of the target deployment to identify a target feature value for the target feature. The recommendation system then provides the identified target feature value as a recommendation for the target feature for the target deployment.

Abstract: A recommendation system for recommending a target feature value for a target feature for a target deployment is provided. The recommendation system, for each of a plurality of deployments, collects feature values for the features of that deployment. The recommendation system then generates a model for recommending a target feature value for the target feature based on the collected feature values of the features for the deployments. The recommendation system applies the model to the features of the target deployment to identify a target feature value for the target feature. The recommendation system then provides the identified target feature value as a recommendation for the target feature for the target deployment.

Abstract: An apparatus to facilitate partially-frozen neural networks for efficient computer vision systems is disclosed. The apparatus includes a frozen core to store fixed weights of a machine learning model, one or more trainable cores coupled to the frozen core, the one or more trainable cores comprising multipliers for trainable weights of the machine learning model, and wherein the alpha blending layer includes a trainable alpha blending parameter, and wherein the trainable alpha blending parameter is a function of a trainable parameter, a sigmoid function, and outputs of frozen and trainable blocks in a preceding layer of the machine learning model.

Abstract: An example apparatus for processing images includes a trainable vision scaler to receive an image. The trainable vision scaler is to generate output including a feature map or an enhanced image based on the image. The trainable vision scaler is to transmit the output to a computer vision network. The computer vision network is trained to perform a computer vision task using the output.

Abstract: In an example method for training image signal processors, a reconstructed image is generated via an image signal processor based on a sensor image. An intermediate loss function is generated based on a comparison of an output of one or more corresponding layers of a computer vision network and a copy of the computer vision network. The output of the computer vision network is based on the reconstructed image. An image signal processor is trained based on the intermediate loss function.

Abstract: An example apparatus for processing images includes a hybrid infinite impulse response-finite impulse response (IIR-FIR) convolution block to receive an image and generate processed image information. The hybrid IIR-FIR convolution block includes a vertical infinite impulse response (IIR) component to approximate a vertical convolution when processing the image.

Abstract: An example apparatus for adjusting eye gaze in images one or more processors to execute instructions to bidirectionally train a neural network; access a target angle and an input image, the input image including an eye in a first position; generate a vector field with the neural network; and generate a gaze-adjusted image based on the vector field, the gaze-adjusted image including the eye in a second position.

Abstract: An example apparatus for correcting gaze in images includes an image receiver to receive an image comprising an eye and a target angle set to a center. The apparatus also includes a bidirectionally trained convolutional neural network (CNN) to receive the image and the target angle from the image receiver and generate a vector field and a brightness map based on the image and the target angle. The apparatus further includes an image corrector to generate a gaze corrected image based on the vector field and the brightness map.

Abstract: In an example method for training image signal processors, a reconstructed image is generated via an image signal processor based on a sensor image. An intermediate loss function is generated based on a comparison of an output of one or more corresponding layers of a computer vision network and a copy of the computer vision network. The output of the computer vision network is based on the reconstructed image. An image signal processor is trained based on the intermediate loss function.

Abstract: Techniques related to computer vision tasks are discussed. Such techniques include applying a pretrained non-linear transform and pretrained details boosting factor to generate an enhanced image from an input image and reducing the bit depth of the enhanced image prior to applying a pretrained computer vision network to perform the computer vision task.

Abstract: In an example method for training image signal processors, a reconstructed image is generated via an image signal processor based on a sensor image. An intermediate loss function is generated based on a comparison of an output of one or more corresponding layers of a computer vision network and a copy of the computer vision network. The output of the computer vision network is based on the reconstructed image. An image signal processor is trained based on the intermediate loss function.

Abstract: An apparatus to facilitate partially-frozen neural networks for efficient computer vision systems is disclosed. The apparatus includes a frozen core to store fixed weights of a machine learning model, one or more trainable cores coupled to the frozen core, the one or more trainable cores comprising multipliers for trainable weights of the machine learning model, and wherein the alpha blending layer includes a trainable alpha blending parameter, and wherein the trainable alpha blending parameter is a function of a trainable parameter, a sigmoid function, and outputs of frozen and trainable blocks in a preceding layer of the machine learning model.

Abstract: A mechanism is described for facilitating automatic tuning of image signal processors using reference images in image processing environments, according to one embodiment. A method of embodiments, as described herein, includes one or more processors to: receive images associated with one or more scenes captured by one or more cameras; access tuning parameters associated with functionalities within an image signal processor (ISP) pipeline; generate reference images based on the tuning parameters, wherein a reference image is associated with an image for each functionality within the ISP pipeline; and automatically tune the ISP pipeline based on selection of one or more of the reference images for one or more of the images for one or more of the functionalities.