Abstract: Technologies for performing sensor fusion include a compute device. The compute device includes circuitry configured to obtain detection data indicative of objects detected by each of multiple sensors of a host system. The detection data includes camera detection data indicative of a two or three dimensional image of detected objects and lidar detection data indicative of depths of detected objects. The circuitry is also configured to merge the detection data from the multiple sensors to define final bounding shapes for the objects.

Abstract: Spatial data may be divided along an axis of the second dimension into a first data segment and a second data segment, such that the first data segment is limited to data points of the spatial data with second dimension coordinates within a first range and the second data segment is limited to data points of the spatial data with second dimension coordinates within a second range. A first processing element may execute an object detection process on the first data segment to generate a first list of objects within the first data segment. A second processing element may execute the object detection process on the second data segment to generate a second list of objects within the second data segment. A first set of objects detected in the first data segment may be combined with a second set of objects detected in the second data segment.

Abstract: Various systems and methods for implementing LiDAR-based object detection and classification are described herein. An object detection system includes a feature extraction and object identification (FEOI) circuit to: receive segmented data of an environment around the object detection system, the segmented data obtained using a light imaging detection and ranging (LiDAR) system, oriented with respect to a direction of travel; compute spatial and structural parameters of a segment of the segmented data; and use the spatial and structural parameters with a machine learning model to obtain a classification of the segment.

Abstract: Spatial data may be divided along an axis of the second dimension into a first data segment and a second data segment, such that the first data segment is limited to data points of the spatial data with second dimension coordinates within a first range and the second data segment is limited to data points of the spatial data with second dimension coordinates within a second range. A first processing element may execute an object detection process on the first data segment to generate a first list of objects within the first data segment. A second processing element may execute the object detection process on the second data segment to generate a second list of objects within the second data segment. A first set of objects detected in the first data segment may be combined with a second set of objects detected in the second data segment.

Abstract: A quaternion deep neural network (QTDNN) includes a plurality of modular hidden layers, each comprising a set of QT computation sublayers, including a quaternion (QT) general matrix multiplication sublayer, a QT non-linear activations sublayer, and a QT sampling sublayer arranged along a forward signal propagation path. Each QT computation sublayer of the set has a plurality of QT computation engines. In each modular hidden layer, a steering sublayer precedes each of the QT computation sublayers along the forward signal propagation path. The steering sublayer directs a forward-propagating quaternion-valued signal to a selected at least one QT computation engine of a next QT computation subsequent sublayer.

Abstract: A machine-learning system includes a quaternion (QT) computation engine. Input data to the QT computation engine includes quaternion values, each comprising a real component and three imaginary components, represented as a set of real-valued tensors. A single quaternion value is represented as a 1-dimensional real-valued tensor having four real-valued components, wherein a first real-valued component represents the real component of the single quaternion value, and wherein a second, a third, and a fourth real-valued component each respectively represents one of the imaginary components. A quaternion-valued vector having a size N is represented as a 2-dimensional real-valued tensor comprising N 1-dimensional real-valued tensors. A quaternion-valued matrix having N×M dimensions is represented as a 3-dimensional real-valued tensor comprising M 2-dimensional real-valued tensors comprising N 1-dimensional real-valued tensors.

Abstract: Spatial data may be divided along an axis of the second dimension into a first data segment and a second data segment, such that the first data segment is limited to data points of the spatial data with second dimension coordinates within a first range and the second data segment is limited to data points of the spatial data with second dimension coordinates within a second range. A first processing element may execute an object detection process on the first data segment to generate a first list of objects within the first data segment. A second processing element may execute the object detection process on the second data segment to generate a second list of objects within the second data segment. A first set of objects detected in the first data segment may be combined with a second set of objects detected in the second data segment.

Abstract: According to one embodiment, an apparatus includes an interface to receive sensor data from a plurality of sensors of an autonomous vehicle. The apparatus also includes processing circuitry to apply a sensor abstraction process to the sensor data to produce abstracted scene data, and to use the abstracted scene data in a perception phase of a control process for the autonomous vehicle. The sensor abstraction process may include one or more of: applying a Sensor data response normalization process to the sensor data, applying a warp process to the sensor data, and applying a filtering process to the sensor data.

Abstract: An apparatus comprising at least one interface to receive sensor data from a plurality of sensors of a vehicle; and one or more processors to autonomously control driving of the vehicle according to a path plan based on the sensor data; determine that autonomous control of the vehicle should cease; send a handoff request to a remote computing system for the remote computing system to control driving of the vehicle remotely; receive driving instruction data from the remote computing system; and control driving of the vehicle based on instructions included in the driving instruction data.

Abstract: Sensor data is received from a plurality of sensors, where the plurality of sensors includes a first set of sensors and a second set of sensors, and at least a portion of the plurality of sensors are coupled to a vehicle. Control of the vehicle is automated based on at least a portion of the sensor data generated by the first set of sensors. Passenger attributes of one or more passengers within the autonomous vehicles are determined from sensor data generated by the second set of sensors. Attributes of the vehicle are modified based on the passenger attributes and the sensor data generated by the first set of sensors.

Abstract: Systems and techniques for neural network training are described herein, a training set may be received for a neural network. Here, the neural network may comprise a set of nodes arranged in layers and a set of inter-node weights between nodes in the set of nodes. The neural network may then be iteratively trained to create a trained neural network. An iteration of the training may include generating a random unit vector and creating an update vector by calculating a magnitude for the random unit vector based on a degree that the random unit vector matches a gradient—where the gradient is represented by a dual number. The iteration may continue by updating a parameter vector for an inter-node weight by subtracting the update vector from a previous parameter vector of the inter-node weight. The trained neural network may then be used to classify data.

Abstract: Technologies for performing sensor fusion include a compute device. The compute device includes circuitry configured to obtain detection data indicative of objects detected by each of multiple sensors of a host system. The detection data includes camera detection data indicative of a two or three dimensional image of detected objects and lidar detection data indicative of depths of detected objects. The circuitry is also configured to merge the detection data from the multiple sensors to define final bounding shapes for the objects.

Abstract: A deep neural network (DNN) includes hidden layers arranged along a forward propagation path between an input layer and an output layer. The input layer accepts training data comprising quaternion values, outputs a quaternion-valued signal along the forward path to at least one of the hidden layers. At least some of the hidden layers include quaternion layers to execute consistent quaternion (QT) forward operations based on one or more variable parameters. A loss function engine produces a loss function representing an error between the DNN result and an expected result. QT backpropagation-based training operations include computing layer-wise QT partial derivatives, consistent with an orthogonal basis of quaternion space, of the loss function with respect to a QT conjugate of the one or more variable parameters and of respective inputs to the quaternion layers.

Abstract: A quaternion deep neural network (QTDNN) includes a plurality of modular hidden layers, each comprising a set of QT computation sublayers, including a quaternion (QT) general matrix multiplication sublayer, a QT non-linear activations sublayer, and a QT sampling sublayer arranged along a forward signal propagation path. Each QT computation sublayer of the set has a plurality of QT computation engines. In each modular hidden layer, a steering sublayer precedes each of the QT computation sublayers along the forward signal propagation path. The steering sublayer directs a forward-propagating quaternion-valued signal to a selected at least one QT computation engine of a next QT computation subsequent sublayer.

Abstract: A deep neural network (DNN) includes hidden layers arranged along a forward propagation path between an input layer and an output layer. The input layer accepts training data comprising quaternion values, outputs a quaternion-valued signal along the forward path to at least one of the hidden layers. At least some of the hidden layers include quaternion layers to execute consistent quaternion (QT) forward operations based on one or more variable parameters. A loss function engine produces a loss function representing an error between the DNN result and an expected result. QT backpropagation-based training operations include computing layer-wise QT partial derivatives, consistent with an orthogonal basis of quaternion space, of the loss function with respect to a QT conjugate of the one or more variable parameters and of respective inputs to the quaternion layers.

Abstract: A machine-learning system includes a quaternion (QT) computation engine. Input data to the QT computation engine includes quaternion values, each comprising a real component and three imaginary components, represented as a set of real-valued tensors. A single quaternion value is represented as a 1-dimensional real-valued tensor having four real-valued components, wherein a first real-valued component represents the real component of the single quaternion value, and wherein a second, a third, and a fourth real-valued component each respectively represents one of the imaginary components. A quaternion-valued vector having a size N is represented as a 2-dimensional real-valued tensor comprising N 1-dimensional real-valued tensors. A quaternion-valued matrix having N×M dimensions is represented as a 3-dimensional real-valued tensor comprising M 2-dimensional real-valued tensors comprising N 1-dimensional real-valued tensors.

Abstract: Various systems and methods for implementing context-based digital signal processing are described herein. An object detection system includes a processor to: access sensor data from a first sensor and a second sensor integrated in a vehicle; access an operating context of the vehicle; assign a first weight to a first object detection result from sensor data of the first sensor, the first weight adjusted based on the operating context; assign a second weight to a second object detection result from sensor data of the second sensor, the second weight adjusted based on the operating context; and perform a combined object detection technique by combining the first object detection result weighted by the first weight and the second object detection result weighted by the second weight.

Abstract: Machine vision processing includes capturing 3D spatial data representing a field of view and including ranging measurements to various points within the field of view, applying a segmentation algorithm to the 3D spatial data to produce a segmentation assessment indicating a presence of individual objects within the field of view, wherein the segmentation algorithm is based on at least one adjustable parameter, and adjusting a value of the at least one adjustable parameter based on the ranging measurements. The segmentation assessment is based on application of the segmentation algorithm to the 3D spatial data, with different values of the at least one adjustable parameter value corresponding to different values of the ranging measurements of the various points within the field of view.

Abstract: Spatial data may be divided along an axis of the second dimension into a first data segment and a second data segment, such that the first data segment is limited to data points of the spatial data with second dimension coordinates within a first range and the second data segment is limited to data points of the spatial data with second dimension coordinates within a second range. A first processing element may execute an object detection process on the first data segment to generate a first list of objects within the first data segment. A second processing element may execute the object detection process on the second data segment to generate a second list of objects within the second data segment. A first set of objects detected in the first data segment may be combined with a second set of objects detected in the second data segment.

Abstract: Machine vision processing includes capturing 3D spatial data representing a field of view and including ranging measurements to various points within the field of view, applying a segmentation algorithm to the 3D spatial data to produce a segmentation assessment indicating a presence of individual objects within the field of view, wherein the segmentation algorithm is based on at least one adjustable parameter, and adjusting a value of the at least one adjustable parameter based on the ranging measurements. The segmentation assessment is based on application of the segmentation algorithm to the 3D spatial data, with different values of the at least one adjustable parameter value corresponding to different values of the ranging measurements of the various points within the field of view.