Abstract: A method for controlling a vehicle includes: receiving, by a controller, route data, wherein the route data is continuously updated while the vehicle is moving, and the vehicle includes a plurality of vehicle operating modes; receiving, by the controller, feature data, wherein the feature data is information about a plurality of features needed for each of the plurality of vehicle operating modes; determining, by the controller, a plurality of ranges for each of the plurality of vehicle operating modes, wherein each of the plurality of ranges is a function of the route data and the feature data for each of the plurality of vehicle operating modes; and commanding, by the controller, a user interface to display a list of range-mode combinations, wherein the list of range-mode combinations includes the plurality of ranges for each of the plurality of vehicle operating modes.

Abstract: Presented are embedded control systems with logic for computation and data sharing, methods for making/using such systems, and vehicles with distributed sensors and embedded processing hardware for provisioning automated driving functionality. A method for operating embedded controllers connected with distributed sensors includes receiving a first data stream from a first sensor via a first embedded controller, and storing the first data stream with a first timestamp and data lifespan via a shared data buffer in a memory device. A second data stream is received from a second sensor via a second embedded controller. A timing impact of the second data stream is calculated based on the corresponding timestamp and data lifespan. Upon determining that the timing impact does not violate a timing constraint, the first data stream is purged from memory and the second data stream is stored with a second timestamp and data lifespan in the memory device.

Abstract: A method of controlling a vehicle includes determining a current operating situation of the vehicle, and identifying a subset of a plurality of sensors of the vehicle needed to provide data to enable a vehicle control function for the current operating situation of the vehicle. A remainder of the plurality of sensors is disengaged to reduce electric energy usage by the vehicle while the vehicle is operating in the current operating situation of the vehicle. A sampling rate for the selected subset of sensors may be reduced to further reduce energy usage of the vehicle. Additionally, an energy reduction processing strategy may be implemented to reduce a processor frequency or a voltage of a computing device used to provide the vehicle control function to further reduce energy usage of the vehicle.

Abstract: A method in a multiprocessor system for processing multiple perception streams is disclosed. The method comprises: reading data from a plurality of perception streams according to a reading schedule determined by a predetermined policy, each perception stream comprising perception data from a different perception sensor; assigning a unique identification tag to each perception stream; writing each perception stream with its unique identification tag to a server input queue based on the predetermined policy; and processing the tagged perception streams using a server. The processing includes: retrieving tagged perception streams from the server input queue; applying a processing algorithm to process the retrieved tagged perception streams; and outputting the processed perception streams to a server output queue.

Abstract: Examples of techniques for using fixed-point quantization in deep neural networks are disclosed. In one example implementation according to aspects of the present disclosure, a computer-implemented method includes capturing a plurality of images at a camera associated with a vehicle and storing image data associated with the plurality of images to a memory. The method further includes dispatching vehicle perception tasks to a plurality of processing elements of an accelerator in communication with the memory. The method further includes performing, by at least one of the plurality of processing elements, the vehicle perception tasks for the vehicle perception using a neural network, wherein performing the vehicle perception tasks comprises quantizing a fixed-point value based on an activation input and a synapse weight. The method further includes controlling the vehicle based at least in part on a result of performing the vehicle perception tasks.

Abstract: Described herein are systems, methods, and computer-readable media for generating and training a high precision low bit convolutional neural network (CNN). A filter of each convolutional layer of the CNN is approximated using one or more binary filters and a real-valued activation function is approximated using a linear combination of binary activations. More specifically, a non-1×1 filter (e.g., a k×k filter, where k>1) is approximated using a scaled binary filter and a 1×1 filter is approximated using a linear combination of binary filters. Thus, a different strategy is employed for approximating different weights (e.g., 1×1 filter vs. a non-1×1 filter). In this manner, convolutions performed in convolutional layer(s) of the high precision low bit CNN become binary convolutions that yield a lower computational cost while still maintaining a high performance (e.g., a high accuracy).

Abstract: A stream manager for managing the distribution of instructions to a plurality of processing devices includes a dispatcher module configured to: receive multiple instruction streams, wherein each instruction stream includes a plurality of requested computations for processing perception data from a perception data source; partition each instruction stream into a plurality of partitions based on type of device to perform a requested computation from the instruction stream; assign a release time and deadline to each partition, and dispatch partition computations to a plurality of scheduling queues to distribute processing of the partition computations amongst the plurality of processing devices. The plurality of scheduling queues include: a plurality of CPU schedulers, wherein each CPU scheduler is assigned to a specific CPU and a specific scheduling queue; and a plurality of accelerator schedulers, wherein each accelerator scheduler is assigned to a specific scheduling queue and a specific type of accelerator.

Abstract: A stream manager for managing the distribution of instructions to a plurality of processing devices includes a dispatcher module configured to: receive multiple instruction streams, wherein each instruction stream includes a plurality of requested computations for processing perception data from a perception data source; partition each instruction stream into a plurality of partitions based on type of device to perform a requested computation from the instruction stream; assign a release time and deadline to each partition, and dispatch partition computations to a plurality of scheduling queues to distribute processing of the partition computations amongst the plurality of processing devices. The plurality of scheduling queues include: a plurality of CPU schedulers, wherein each CPU scheduler is assigned to a specific CPU and a specific scheduling queue; and a plurality of accelerator schedulers, wherein each accelerator scheduler is assigned to a specific scheduling queue and a specific type of accelerator.

Abstract: A method for controlling a vehicle includes: receiving, by a controller, route data, wherein the route data is continuously updated while the vehicle is moving, and the vehicle includes a plurality of vehicle operating modes; receiving, by the controller, feature data, wherein the feature data is information about a plurality of features needed for each of the plurality of vehicle operating modes; determining, by the controller, a plurality of ranges for each of the plurality of vehicle operating modes, wherein each of the plurality of ranges is a function of the route data and the feature data for each of the plurality of vehicle operating modes; and commanding, by the controller, a user interface to display a list of range-mode combinations, wherein the list of range-mode combinations includes the plurality of ranges for each of the plurality of vehicle operating modes.

Abstract: Examples of techniques for dynamically selecting a batch size used in vehicle camera image processing are disclosed. In one example implementation, a method includes generating, by a processing device, a batch table and a mode table. The method further includes determining, by the processing device, image processing performance requirements for a current mode of a vehicle using the mode table, the vehicle comprising a plurality of cameras configured to capture a plurality of images. The method further includes selecting, by the processing device, a batch size and a processing frequency based at least in part on the image processing performance requirements for the current mode of the vehicle. The method further includes processing, by an accelerator, at least a subset of the plurality of images based at least in part on the batch size and processing frequency.

Abstract: A method and system including a central processing unit (CPU), an accelerator, a communication bus and a system memory device for dynamically processing an image file are described. The accelerator includes a local memory buffer, a data transfer scheduler, and a plurality of processing engines. The data transfer scheduler is arranged to manage data transfer between the system memory device and the local memory buffer, wherein the data transfer includes data associated with the image file. The local memory buffer is configured as a circular line buffer, and the data transfer scheduler includes a ping-pong buffer for transferring output data from the one of the processing engines to the system memory device. The local memory buffer is configured to execute cross-layer usage of data associated with the image file.

Abstract: A signal processing system includes a central processing unit (CPU) in communication with an accelerator, and an instruction scheduler in communication with the accelerator. A first memory device including a first instruction set is configured to operate the accelerator, a second instruction set is configured to operate the CPU, and a second memory device is configured to receive a datafile. The accelerator includes a plurality of processing engines (PEs) and an instruction scheduler, the instruction set includes a plurality of operators, and the instruction scheduler is configured to implement the operators in the accelerator employing the PEs. The CPU employs the operators implemented in the accelerator to analyze the datafile to extract a feature therefrom.

Abstract: A method and system including a central processing unit (CPU), an accelerator, a communication bus and a system memory device for dynamically processing an image file are described. The accelerator includes a local memory buffer, a data transfer scheduler, and a plurality of processing engines. The data transfer scheduler is arranged to manage data transfer between the system memory device and the local memory buffer, wherein the data transfer includes data associated with the image file. The local memory buffer is configured as a circular line buffer, and the data transfer scheduler includes a ping-pong buffer for transferring output data from the one of the processing engines to the system memory device. The local memory buffer is configured to execute cross-layer usage of data associated with the image file.

Abstract: In one example implementation according to aspects of the present disclosure, a computer-implemented method includes capturing a plurality of images at a camera associated with a vehicle and storing image data associated with the plurality of images to a memory. The method further includes dispatching vehicle perception tasks to a plurality of processing elements of an accelerator in communication with the memory. The method further includes performing, by at least one of the plurality of processing elements, the vehicle perception tasks for the vehicle perception using a neural network, wherein performing the vehicle perception tasks comprises performing an activation bypass for values below a first threshold, and performing weight pruning of synapses and neurons of the neural network based at least in part on a second threshold. The method further includes controlling the vehicle based at least in part on a result of performing the vehicle perception tasks.

Abstract: Examples of techniques for using fixed-point quantization in deep neural networks are disclosed. In one example implementation according to aspects of the present disclosure, a computer-implemented method includes capturing a plurality of images at a camera associated with a vehicle and storing image data associated with the plurality of images to a memory. The method further includes dispatching vehicle perception tasks to a plurality of processing elements of an accelerator in communication with the memory. The method further includes performing, by at least one of the plurality of processing elements, the vehicle perception tasks for the vehicle perception using a neural network, wherein performing the vehicle perception tasks comprises quantizing a fixed-point value based on an activation input and a synapse weight. The method further includes controlling the vehicle based at least in part on a result of performing the vehicle perception tasks.

Abstract: Described herein are systems, methods, and computer-readable media for generating and training a high precision low bit convolutional neural network (CNN). A filter of each convolutional layer of the CNN is approximated using one or more binary filters and a real-valued activation function is approximated using a linear combination of binary activations. More specifically, a non-1×1 filter (e.g., a k×k filter, where k>1) is approximated using a scaled binary filter and a 1×1 filter is approximated using a linear combination of binary filters. Thus, a different strategy is employed for approximating different weights (e.g., 1×1 filter vs. a non-1×1 filter). In this manner, convolutions performed in convolutional layer(s) of the high precision low bit CNN become binary convolutions that yield a lower computational cost while still maintaining a high performance (e.g., a high accuracy).

Abstract: An adaptive parallel imaging processing system in a vehicle is provided. The system may include, but is not limited to, a plurality of processors and a resource management system including, but not limited to, an execution monitor, the execution monitor configured to calculate an average utilization of each of the plurality of processors over a moving window, and a service scheduler controlling a request queue for each of the plurality of processors, the service scheduler scheduling image processing tasks in the respective request queue for the each of the plurality of processors based upon the average utilization of each of the plurality of processors, the capabilities of each of the plurality of processors, and a priority associated with each image processing task, wherein an autonomous vehicle control system is configured to generate the instructions to control the at least one vehicle system based upon the processed image processing tasks.

Abstract: Systems and methods are provided for testing a first computer device of a vehicle. A method includes selecting an operational component of the first computer device and selecting a test operation that is configured to utilize an entire capacity of the operational component. The method further includes instructing the first computer device to perform the test operation and to generate a first result. The method further yet includes retrieving a second result of the test operation and comparing the first result of the test operation from the first computer device with the second result. The method further yet includes indicating that the first computer device is faulty based at least in part on a difference between the first result and the second result.

Abstract: Examples of techniques for dynamically selecting a batch size used in vehicle camera image processing are disclosed. In one example implementation, a method includes generating, by a processing device, a batch table and a mode table. The method further includes determining, by the processing device, image processing performance requirements for a current mode of a vehicle using the mode table, the vehicle comprising a plurality of cameras configured to capture a plurality of images. The method further includes selecting, by the processing device, a batch size and a processing frequency based at least in part on the image processing performance requirements for the current mode of the vehicle. The method further includes processing, by an accelerator, at least a subset of the plurality of images based at least in part on the batch size and processing frequency.

Abstract: A method of controlling a vehicle includes determining a current operating situation of the vehicle, and identifying a subset of a plurality of sensors of the vehicle needed to provide data to enable a vehicle control function for the current operating situation of the vehicle. A remainder of the plurality of sensors is disengaged to reduce electric energy usage by the vehicle while the vehicle is operating in the current operating situation of the vehicle. A sampling rate for the selected subset of sensors may be reduced to further reduce energy usage of the vehicle. Additionally, an energy reduction processing strategy may be implemented to reduce a processor frequency or a voltage of a computing device used to provide the vehicle control function to further reduce energy usage of the vehicle.