Abstract: Reconfigurable shared memory systems, and related processor-based systems and methods are disclosed. The reconfigurable shared memory system can be included in a processor-based system to provide memory for data storage. In exemplary aspects, the reconfigurable shared memory system not only includes the dedicated memory and the general memory (e.g., system cache memory), but also includes a reconfigurable memory. The reconfigurable memory can be configured as either part of addressable memory space of the dedicated memory if an application requires such additional dedicated memory, and/or configured as part of the addressable memory space of the general memory to provide additional memory to other clients for increased processing performance if such reconfigurable memory is not needed as part of the dedicated memory. The dedicated memory does not have to be sized to the worst-case size requirements of a given application.

Abstract: Examples are described for applying different settings for image capture to different portions of image data. For example, an image sensor can capture image data of a scene and can send the image data to an image signal processor (ISP) and a classification engine for processing. The classification engine can determine that a first object image region depicts a first category of object, and a second object image region depicts a second category of object. Different confidence regions of the image data can identify different degrees of confidence in the classifications. The ISP can generate an image by applying a different settings to the different portions of the image data. The different portions of the image data can be identified based on the object image regions and confidence regions.

Abstract: Examples are described for applying different settings for image capture to different portions of image data. For example, an image sensor can capture image data of a scene and can send the image data to an image signal processor (ISP) and a classification engine for processing. The classification engine can determine that a first object image region depicts a first category of object, and a second object image region depicts a second category of object. Different confidence regions of the image data can identify different degrees of confidence in the classifications. The ISP can generate an image by applying a different settings to the different portions of the image data. The different portions of the image data can be identified based on the object image regions and confidence regions.

Abstract: A method of generating a set of examples for explaining decisions made by a machine learning program, involving receiving a set of training data for training the program, and for given subsets of the training data, determining each of (a) a probability of a user correctly inferring a future decision of the program after observing the respective decisions of the program for the given subset of the training data, (b) a suitability of a size of the given subset, and (c) an average probability of the user correctly inferring a future decision of the program after observing the respective decisions of the program for an unspecified subset of the training data. The determinations (a), (b) and (c) are used to score each of the given subsets of training data, and a subset of training data is selected as the generated set of examples based on the scores.

Abstract: Aspects relate to an image signal processor that processes frames at changing frame rates. An example method includes receiving, by an image signal processor, a first sequence of image frames from an image sensor at a first frame rate, processing each image frame of the first sequence of image frames at the first frame rate, and receiving from the image sensor an indication of a frame rate change from the first frame rate to a second frame rate. The method also includes configuring one or more filters of the image signal processor to process image frames from the image sensor in response to receiving the indication of the frame rate change from the image sensor, receiving a second sequence of image frames from the image sensor at the second frame rate, and processing each image frame of the second sequence of image frames at the second frame rate.

Abstract: Aspects relate to an image signal processor that processes frames from different image sensors. An example device includes a memory and an image signal processor coupled to the memory. The image signal processor is configured to provide a first trigger to a first image sensor (the first image sensor being coupled to the image signal processor), receive a first frame from the first image sensor at a first time in response to the first trigger being received by the first image sensor, process the first frame, provide a second trigger to the second image sensor (the second image sensor being coupled to the image signal processor), receive a second frame from the second image sensor at a second time in response to the second trigger being received by the second image sensor (with the second time subsequent to the first time), and process the second frame.

Abstract: Techniques and systems are provided for processing image data. For example, an image signal processor can obtain (e.g., from a host processor) a first setting change indicator value indicating a change in parameter settings of the image signal processor. The image signal processor can obtain an image frame from an image sensor, and can determine a second setting change indicator value from the image frame. The second setting change indicator value can be provided to the image sensor from the host processor. The second setting change indicator value indicates a change in parameter settings of the image sensor. The image signal processor can compare the first setting change indicator value to the second setting change indicator value, and can determine whether to process the image frame or to drop the image frame based on comparing the first setting change indicator value to the second setting change indicator value.

Abstract: Examples are described for applying different settings for image capture to different portions of image data. For example, an image sensor can capture image data of a scene and can send the image data to an image signal processor (ISP) and a classification engine for processing. The classification engine can determine that a first object image region depicts a first category of object, and a second object image region depicts a second category of object. Different confidence regions of the image data can identify different degrees of confidence in the classifications. The ISP can generate an image by applying a different settings to the different portions of the image data. The different portions of the image data can be identified based on the object image regions and confidence regions.

Abstract: Aspects relate to an image signal processor that processes frames at changing frame rates. An example method includes receiving, by an image signal processor, a first sequence of image frames from an image sensor at a first frame rate, processing each image frame of the first sequence of image frames at the first frame rate, and receiving from the image sensor an indication of a frame rate change from the first frame rate to a second frame rate. The method also includes configuring one or more filters of the image signal processor to process image frames from the image sensor in response to receiving the indication of the frame rate change from the image sensor, receiving a second sequence of image frames from the image sensor at the second frame rate, and processing each image frame of the second sequence of image frames at the second frame rate.

Abstract: Techniques and systems are provided for processing image data. For example, an image signal processor can obtain (e.g., from a host processor) a first setting change indicator value indicating a change in parameter settings of the image signal processor. The image signal processor can obtain an image frame from an image sensor, and can determine a second setting change indicator value from the image frame. The second setting change indicator value can be provided to the image sensor from the host processor. The second setting change indicator value indicates a change in parameter settings of the image sensor. The image signal processor can compare the first setting change indicator value to the second setting change indicator value, and can determine whether to process the image frame or to drop the image frame based on comparing the first setting change indicator value to the second setting change indicator value.

Abstract: Systems, methods, and non-transitory media are provided for reducing resource and power usage and requirements in staggered high dynamic range (HDR) applications. For example, a first exposure including a set of image data associated with a frame can be stored in memory. The first exposure has a first exposure time and is captured by an image sensor during a first time period associated with the frame. The first exposure can be obtained from the memory, and a second exposure including a set of image data associated with the frame can be obtained from a cache or the image sensor. The second exposure has a second exposure time and is captured by the image sensor during a second time period associated with the frame. The sets of image data from the first and second exposures can be merged, and an HDR image generated based on the sets of image data merged.

Abstract: A method of generating a set of examples for explaining decisions made by a machine learning program, involving receiving a set of training data for training the program, and for given subsets of the training data, determining each of (a) a probability of a user correctly inferring a future decision of the program after observing the respective decisions of the program for the given subset of the training data, (b) a suitability of a size of the given subset, and (c) an average probability of the user correctly inferring a future decision of the program after observing the respective decisions of the program for an unspecified subset of the training data. The determinations (a), (b) and (c) are used to score each of the given subsets of training data, and a subset of training data is selected as the generated set of examples based on the scores.

Abstract: Methods and apparatus to manage image signal processor (ISP) data traffic is provided. The apparatus includes an ISP having an ISP front-end configured to receive image data and a first memory configured to store the image data. The ISP front-end is further configured to output the image data stored in the first memory to a second memory via a memory link in response to the image data stored in the first memory reaching a size threshold. Another apparatus includes apparatus includes a camera sensor configured to output image data in a camera mode, an ISP on a die, a camera link coupling the camera sensor and the ISP, a memory, and a memory link coupling the ISP and the memory. The memory link is configured to enter a low-power mode in the camera mode.

Abstract: Methods, systems, and devices for image processing are described. A device may include a plurality of buffer components, each of which may receive a pixel lines that may each be associated with a respective raw image. An arbitration component of the device may combine at least some of the pixel lines into one or more data packets. The arbitration component may pass, using an arbitration scheme such as a time division multiplexing scheme, the one or more data packets from the arbitration component to a shared image signal processor (ISP) of the device. The shared ISP may generate a respective processed image based at least in part on the one or more data packets. In some examples, the device may maintain a respective set of image statistics, registers, and the like for at least some of the raw images.

Abstract: Transmission of data over a serial link based on a unidirectional clock signal is provided. A unidirectional clock signal is generated based on a first clock of a master device. The unidirectional clock signal is sent to a slave device that is connected to the serial link. The master device transmits data to the slave device over the serial link based on the first clock. The slave device receives the unidirectional clock signal from a master device. The slave device transmits data over the serial link to the master device based on the unidirectional clock signal.

Abstract: Methods and apparatus to manage image signal processor (ISP) data traffic is provided. The apparatus includes an ISP having an ISP front-end configured to receive image data and a first memory configured to store the image data. The ISP front-end is further configured to output the image data stored in the first memory to a second memory via a memory link in response to the image data stored in the first memory reaching a size threshold. Another apparatus includes apparatus includes a camera sensor configured to output image data in a camera mode, an ISP on a die, a camera link coupling the camera sensor and the ISP, a memory, and a memory link coupling the ISP and the memory. The memory link is configured to enter a low-power mode in the camera mode.

Abstract: A serial transceiver that includes programmable distributed data processing is provided. The serial transceiver can include an ingress channel that receives serial ingress data and an egress channel that transmits serial egress data. The serial transceiver can also include first and second layers that are one and another of a transport layer, a link layer, or a physical layer (PHY). The first and second layers can include elements that process the ingress data and the egress data. The serial transceiver can also include a programmable controller, a first interconnect that connects the programmable controller to the first layer, and a second interconnect that connects the programmable controller to the second layer. The programmable controller can send first data via the first interconnect to the first layer, and the first data can be processed by one of the first layer elements.

Abstract: Methods and apparatus improve static region detection in an imaging pipeline. An imaging pipeline may perform detection of static regions of an image at multiple stages of the pipeline. For example, as static regions may be eliminated from further processing by the imaging pipeline, static region detection performed at an early stage of the pipeline may provide for maximized power savings. As images early in the pipeline may contain artifacts inhibiting detection of some static regions, additional static region detection may be performed after further image processing. For example, static region detection may be performed for a second time after some filtering is applied to images in the pipeline. Regions previously characterized as dynamic may be characterized as static later in the pipeline due to a reduction of noise for example provided by the filters, and differences between the static region detection at different positions within the imaging pipeline.

Abstract: Transmission of data over a serial link based on a unidirectional clock signal is provided. A unidirectional clock signal is generated based on a first clock of a master device. The unidirectional clock signal is sent to a slave device that is connected to the serial link. The master device transmits data to the slave device over the serial link based on the first clock. The slave device receives the unidirectional clock signal from a master device. The slave device transmits data over the serial link to the master device based on the unidirectional clock signal.

Abstract: Serial communication using a packetization protocol engineered for efficient transmission is provided. Data link layer (DLL) control packets can be generated for transmission of control messages. Each DLL control message packet can have a DLL control packet length, and the DLL control packet length can be a fixed length. Physical layer (PHY) control packets can be generated. Each PHY control packet can include one of the DLL control packets and a control token. The length of each PHY control packet can be the sum of the DLL control packet length and a control token length of the control token. The PHY control packets can be encapsulated in frames. Each of the frames can include a synchronization symbol having a symbol length. The length of each frame can be the sum of the symbol length and an encapsulation length, which can be twice the length of the PHY control packet.