Abstract: An N×N multiplier may include a N/2×N first multiplier, a N/2×N/2 second multiplier, and a N/2×N/2 third multiplier. The N×N multiplier receives two operands to multiply. The first, second and/or third multipliers are selectively disabled if an operand equals zero or has a small value. If the operands are both less than 2N/2, the second or the third multiplier are used to multiply the operands. If one operand is less than 2N/2 and the other operand is equal to or greater than 2N/2, the first multiplier is used or the second and third multipliers are used to multiply the operands. If both operands are equal to or greater than 2N/2, the first, second and third multipliers are used to multiply the operands.

Abstract: A system and a method of quantizing a pre-trained neural network, includes determining by a layer/channel bit-width determiner for each layer or channel of the pre-trained neural network a minimum quantization noise for the layer or the channel for each master bit-width value in a predetermined set of master bit-width values; and selecting by a bit-width selector for the layer or the channel the master bit-width value having the minimum quantization noise for the layer or the channel. In one embodiment, the minimum quantization noise for the layer or the channel is based on a square of a range of weights for the layer or the channel that is multiplied by a constant to a negative power of a current master bit-width value.

Abstract: The Time-of-Flight (TOF) technique is combined with analog amplitude modulation within each pixel in an image sensor. The pixel may be a two-tap pixel or a one-tap pixel. Two photoelectron receiver circuits in the pixel receive respective analog modulating signals. The distribution of the received photoelectron charge between these two circuits is controlled by the difference (or ratio) of the two analog modulating voltages. The differential signals generated in this manner within the pixel are modulated in time domain for TOF measurement. Thus, the TOF information is added to the received light signal by the analog domain-based single-ended to differential converter inside the pixel itself. The TOF-based measurement of range and its resolution are controllable by changing the duration of modulation. An autonomous navigation system with these features may provide improved vision for drivers under difficult driving conditions like low light, fog, bad weather, or strong ambient light.

Abstract: A Dynamic Vision Sensor (DVS) pose-estimation system includes a DVS, a transformation estimator, an inertial measurement unit (IMU) and a camera-pose estimator based on sensor fusion. The DVS detects DVS events and shapes frames based on a number of accumulated DVS events. The transformation estimator estimates a 3D transformation of the DVS camera based on an estimated depth and matches confidence-level values within a camera-projection model such that at least one of a plurality of DVS events detected during a first frame corresponds to a DVS event detected during a second subsequent frame. The IMU detects inertial movements of the DVS with respect to world coordinates between the first and second frames. The camera-pose estimator combines information from a change in a pose of the camera-projection model between the first frame and the second frame based on the estimated transformation and the detected inertial movements of the DVS.

Abstract: An image sensor includes a time-resolving sensor and a processor. The time-resolving sensor outputs a first signal and a second signal pair in response detecting one or more photons that have been reflected from an object. A first ratio of a magnitude of the first signal to a sum of the magnitude of the first signal and a magnitude of the second signal is proportional to a time of flight of the one or more detected photons. A second ratio of the magnitude of the second signal to the sum of the magnitude of the first signal and the magnitude of the second signal is proportional to the time of flight of the one or more detected photons. The processor determines a surface reflectance of the object where the light pulse has been reflected based on the first signal and the second signal pair and may generate a grayscale image.

Abstract: An apparatus and a method are provided. The apparatus includes a light source configured to project light in a changing pattern that reduces the light's noticeability; collection optics through which light passes and forms an epipolar plane with the light source; and an image sensor configured to receive light passed through the collection optics to acquire image information and depth information simultaneously. The method includes projecting light by a light source in a changing pattern that reduces the light's noticeability; passing light through collection optics and forming an epipolar plane between the collection optics and the light source; and receiving in an image sensor light passed through the collection optics to acquire image information and depth information simultaneously.

Abstract: A neural processor. In some embodiments, the processor includes a first tile, a second tile, a memory, and a bus. The bus may be connected to the memory, the first tile, and the second tile. The first tile may include: a first weight register, a second weight register, an activations buffer, a first multiplier, and a second multiplier. The activations buffer may be configured to include: a first queue connected to the first multiplier and a second queue connected to the second multiplier. The first queue may include a first register and a second register adjacent to the first register, the first register being an output register of the first queue. The first tile may be configured: in a first state: to multiply, in the first multiplier, a first weight by an activation from the output register of the first queue, and in a second state: to multiply, in the first multiplier, the first weight by an activation from the second register of the first queue.

Abstract: A neural processor. In some embodiments, the processor includes a first tile, a second tile, a memory, and a bus. The bus may be connected to the memory, the first tile, and the second tile. The first tile may include: a first weight register, a second weight register, an activations buffer, a first multiplier, and a second multiplier. The activations buffer may be configured to include: a first queue connected to the first multiplier and a second queue connected to the second multiplier. The first queue may include a first register and a second register adjacent to the first register, the first register being an output register of the first queue. The first tile may be configured: in a first state: to multiply, in the first multiplier, a first weight by an activation from the output register of the first queue, and in a second state: to multiply, in the first multiplier, the first weight by an activation from the second register of the first queue.

Abstract: A neural processor. In some embodiments, the processor includes a first tile, a second tile, a memory, and a bus. The bus may be connected to the memory, the first tile, and the second tile. The first tile may include: a first weight register, a second weight register, an activations buffer, a first multiplier, and a second multiplier. The activations buffer may be configured to include: a first queue connected to the first multiplier and a second queue connected to the second multiplier. The first queue may include a first register and a second register adjacent to the first register, the first register being an output register of the first queue. The first tile may be configured: in a first state: to multiply, in the first multiplier, a first weight by an activation from the output register of the first queue, and in a second state: to multiply, in the first multiplier, the first weight by an activation from the second register of the first queue.

Abstract: Using one or more patterned markers inside the projector module of a three-dimensional (3D) camera to facilitate automatic calibration of the camera's depth sensing operation. The 3D camera utilizes epipolar geometry-based imaging in conjunction with laser beam point-scans in a triangulation-based approach to depth measurements. A light-sensing element and one or more reflective markers inside the projector module facilitate periodic self-calibration of camera's depth sensing operation. To calibrate the camera, the markers are point-scanned using the laser beam and the reflected light is sensed using the light-sensing element. Based on the output of the light-sensing element, the laser's turn-on delay is adjusted to perfectly align a laser light spot with the corresponding reflective marker. Using reflective markers, the exact direction and speed of the scanning beam over time can be determined as well.

Abstract: A client device configured with a neural network includes a processor, a memory, a user interface, a communications interface, a power supply and an input device, wherein the memory includes a trained neural network received from a server system that has trained and configured the neural network for the client device. A server system and a method of training a neural network are disclosed.

Abstract: A neural processor. In some embodiments, the processor includes a first tile, a second tile, a memory, and a bus. The bus may be connected to the memory, the first tile, and the second tile. The first tile may include: a first weight register, a second weight register, an activations buffer, a first multiplier, and a second multiplier. The activations buffer may be configured to include: a first queue connected to the first multiplier and a second queue connected to the second multiplier. The first queue may include a first register and a second register adjacent to the first register, the first register being an output register of the first queue. The first tile may be configured: in a first state: to multiply, in the first multiplier, a first weight by an activation from the output register of the first queue, and in a second state: to multiply, in the first multiplier, the first weight by an activation from the second register of the first queue.

Abstract: A Dynamic Vision Sensor (DVS) pose-estimation system includes a DVS, a transformation estimator, an inertial measurement unit (IMU) and a camera-pose estimator based on sensor fusion. The DVS detects DVS events and shapes frames based on a number of accumulated DVS events. The transformation estimator estimates a 3D transformation of the DVS camera based on an estimated depth and matches confidence-level values within a camera-projection model such that at least one of a plurality of DVS events detected during a first frame corresponds to a DVS event detected during a second subsequent frame. The IMU detects inertial movements of the DVS with respect to world coordinates between the first and second frames. The camera-pose estimator combines information from a change in a pose of the camera-projection model between the first frame and the second frame based on the estimated transformation and the detected inertial movements of the DVS.

Abstract: Using the same image sensor to capture a two-dimensional (2D) image and three-dimensional (3D) depth measurements for a 3D object. A laser point-scans the surface of the object with light spots, which are detected by a pixel array in the image sensor to generate the 3D depth profile of the object using triangulation. Each row of pixels in the pixel array forms an epipolar line of the corresponding laser scan line. Timestamping provides a correspondence between the pixel location of a captured light spot and the respective scan angle of the laser to remove any ambiguity in triangulation. An Analog-to-Digital Converter (ADC) in the image sensor operates as a Time-to-Digital (TDC) converter to generate timestamps. A timestamp calibration circuit is provided on-board to record the propagation delay of each column of pixels in the pixel array and to provide necessary corrections to the timestamp values generated during 3D depth measurements.

Abstract: A structured-light imaging system includes a projector, an image sensor and a controller. The projector projects a structured-light pattern onto a selected slice of a scene in which the selected slice of the scene includes a first predetermined size in a first direction and a second predetermined size in a second direction that is substantially orthogonal to the first direction. The image sensor scans the selected slice of the scene and generates an output corresponding to each region of at least one region of the selected slice. The image sensor and the projector are synchronized in an epipolar manner. The controller is coupled to the image sensor and detects whether an object is located within each scanned region and controls the projector to project the structured-light pattern a first plurality of times towards regions of the selected slice of the scene in which no object has been detected.

Abstract: An image signal processing (ISP) system is provided. The system includes a neural network trained by inputting a set of raw data images and a correlating set of desired quality output images; the neural network including an input for receiving input image data and providing processed output; wherein the processed output includes input image data that has been adjusted for at least one image quality attribute. A method and an imaging device are disclosed.

Abstract: Using the same image sensor to capture a two-dimensional (2D) image and three-dimensional (3D) depth measurements for a 3D object. A laser point-scans the surface of the object with light spots, which are detected by a pixel array in the image sensor to generate the 3D depth profile of the object using triangulation. Each row of pixels in the pixel array forms an epipolar line of the corresponding laser scan line. Timestamping provides a correspondence between the pixel location of a captured light spot and the respective scan angle of the laser to remove any ambiguity in triangulation. An Analog-to-Digital Converter (ADC) in the image sensor operates as a Time-to-Digital (TDC) converter to generate timestamps. A timestamp calibration circuit is provided on-board to record the propagation delay of each column of pixels in the pixel array and to provide necessary corrections to the timestamp values generated during 3D depth measurements.

Abstract: A method for configuring a neural network is provided. The method includes: selecting a neural network including a plurality of layers, each of the layers including a plurality of neurons for processing an input and providing an output; and, incorporating at least one switch configured to randomly select and disable at least a portion of the neurons in each layer. Another method in the computer program product is disclosed.

Abstract: A client device configured with a neural network includes a processor, a memory, a user interface, a communications interface, a power supply and an input device, wherein the memory includes a trained neural network received from a server system that has trained and configured the neural network for the client device. A server system and a method of training a neural network are disclosed.

Abstract: A Time-of-Flight (TOF) technique is combined with analog amplitude modulation within each pixel in a pixel array using multiple Single Photon Avalanche Diodes (SPADs) in conjunction with a single Pinned Photo Diode (PPD) in each pixel. A SPAD may be shared among multiple neighboring pixels. The TOF information is added to the received light signal by the analog domain-based single-ended to differential converter inside the pixel itself. The spatial-temporal correlation among outputs of multiple, adjacent SPADs in a pixel is used to control the operation of the PPD to facilitate recording of TOF values and range of an object. Erroneous range measurements due to ambient light are prevented by stopping the charge transfer from the PPD—and, hence, recording a TOF value—only when two or more SPADs in the pixel are triggered within a pre-defined time interval. An autonomous navigation system with multi-SPAD pixels provides improved vision for drivers under difficult driving conditions.