Abstract: Techniques for detecting and responding to an emergency vehicle are discussed. A vehicle computing system may determine that an emergency vehicle based on sensor data, such as audio and visual data. In some examples, the vehicle computing system may determine aggregate actions of objects (e.g., other vehicles yielding) proximate the vehicle based on the sensor data. In such examples, a determination that the emergency vehicle is operating may be based on the actions of the objects. The vehicle computing system may, in turn, identify a location to move out of a path of the emergency vehicle (e.g., yield) and may control the vehicle to the location. The vehicle computing system may determine that the emergency vehicle is no longer relevant to the vehicle and may control the vehicle along a route to a destination. Determining to yield and/or returning to a mission may be confirmed by a remote operator.

Abstract: A vehicle computing system may implement techniques to dynamically adjust a volume and/or frequency of a sound emitted from a vehicle to warn an object (e.g., dynamic object) of a potential conflict with the vehicle. The techniques may include determining a baseline noise level and/or frequencies proximate to the object. The baseline noise level and/or frequencies may be determined based on an identification of one or more noise generating objects in the environment. The vehicle computing system may determine the volume and/or a frequency of the sound based in part on the baseline noise level and/or frequencies, an urgency of the warning, a probability of conflict between the vehicle and the object, a speed of the object, etc.

Abstract: Techniques for determining a direction of arrival of an emergency are discussed. A plurality of audio sensors of a vehicle can receive audio data associated with the vehicle. An audio sensor pair can be selected from the plurality of audio sensors to generate audio data representing sound in an environment of the vehicle. An angular spectrum associated with the audio sensor pair can be determined based on the audio data. A feature associated with the audio data can be determined based on the angular spectrum and/or the audio data itself. A direction of arrival (DoA) value associated with the sound may be determined based on the feature using a machine learned model. An emergency sound (e.g., a siren) can be detected in the audio data and a direction associated with the emergency relative to the vehicle can be determined based on the feature and the DoA value.

Abstract: Techniques for using beam-formed acoustic notifications for pedestrian notification are described. Computing device(s) can receive sensor data associated with an object in an environment of a vehicle. The computing device(s) can determine first data for emitting a first beam of acoustic energy via speaker(s) of an acoustic array associated with the vehicle, and second data for emitting a second beam of acoustic energy via speakers of the acoustic array. The computing device(s) can cause the speaker(s) to emit the first beam in a direction of the object at a first time and the second beam in the direction of the object at a second time. Directions of propagation of the first beam and the second beam are offset so that the object can localize the source the acoustic notification, thereby localizing the vehicle in the environment.

Abstract: Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. Sensor data generated by the time-of-flight sensor can be impacted by glare. In some examples, corrected data is generated by quantifying glare. A glare region including pixels that are not associated with an object in a range of the time-of-flight sensor may provide glare intensity and glare depth values used to quantify the glare. The glare intensity and glare depth may be used to correct measured data.

Abstract: Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. Sensor data generated by the time-of-flight sensor can return pixels subject to over-exposure or saturation, which may be from stray light. In some examples, multiple exposures captured at different exposure times can be used to determine a saturation value for sensor data. The saturation value may be used to determine a threshold intensity against which intensity values of a primary exposure are compared. A filtered data set can be obtained based on the comparison.

Abstract: Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. Sensor data generated by the time-of-flight sensor can return unreliable pixels, e.g., in the case of over-exposure or saturation. In some examples, multiple exposures captured at different exposure times can be used to determine an overall saturation value or metric representative of the sensor data. The saturation value may be used to control parameters of the sensor. For instance, the saturation value may be used to determine power control parameters for the sensor, e.g., to reduce over- and/or under-exposure.

Abstract: A LIDAR sensor assembly includes a laser light source to emit laser light, and a light sensor to produce a light signal in response to sensing reflections of the laser light emitted by the laser light source from a reference surface that is fixed in relation to the LIDAR sensor assembly. A controller of the LIDAR sensor assembly can process a plurality of samples of reflected light signals, process the samples to remove erroneous readings, and then provide accurate distance measurement. The system can use low-pass filters, or other components, to filter the plurality of samples to enable the “actual,” or primary, reflected light signal (i.e., light signal reflected off of a surface in an environment external to the sensor assembly, as opposed to extraneous, internal reflections off of lenses or other components or noise) to be identified and an accurate time of flight to be calculated.

Abstract: A LIDAR device can accurately calculate distances to objects in an environment by classifying a signal received from a sensor as being a particular type of signal (e.g., saturated or unsaturated) and selecting, based on the type of signal, a detector for processing the received signal from among multiple detectors. For example, the multiple detectors may include different programming and/or circuitry for determining a time delay of arrival (TDOA) between a time that a light pulse was emitted to a time that a pulse reflected off an object was received at a light sensor. The output of the selected detector may then be used to calculate a distance to the object from which the received signal was reflected.

Abstract: Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. The sensor may generate first image data at a first configuration and second image data at a second configuration. The first image data and the second image data may be combined to provide disambiguated depth and improved intensity values for imaging the environment. In some examples, the first and second configurations may have different modulation frequencies, different integration times, and/or different illumination intensities. In some examples, configurations may be dynamically altered based on depth and/or intensity information of a previous frame.

Abstract: A device can accurately discriminate an active pulse from noise by setting a dynamic noise floor that adjusts according to environmental conditions. For example, the device may discriminate, as the active pulse, a light pulse emitted by a light emitter of the system and reflected off an object to a light sensor, from noise such as sunlight glare by determining a dynamic noise floor and identifying, as an active pulse, at least a portion of the received signal that exceeds the dynamic noise floor for a threshold number of samples. The dynamic noise floor may be determined, for example, using a moving average of the received signal and/or shifting or scaling the noise floor based on other properties of the return signal.

Abstract: Techniques for using beam-formed acoustic notifications for pedestrian notification are described. Computing device(s) can receive sensor data associated with an object in an environment of a vehicle. The computing device(s) can determine first data for emitting a first beam of acoustic energy via speaker(s) of an acoustic array associated with the vehicle, and second data for emitting a second beam of acoustic energy via speakers of the acoustic array. The computing device(s) can cause the speaker(s) to emit the first beam in a direction of the object at a first time and the second beam in the direction of the object at a second time. Directions of propagation of the first beam and the second beam are offset so that the object can localize the source the acoustic notification, thereby localizing the vehicle in the environment.

Abstract: Particulate matter, such as dust, steam, smoke, rain, etc. may cause one or more sensor types to generate false positive detections. In particular, various depth measurements may be impeded by particulate matter. Identifying a false return and/or removing a false detection based at least in part on a sensor output may comprise determining a similarity of a portion of a return signal to an emitted light pulse or an expected return signal, determining a variance of the signal portion over time, determining a difference between a power spectrum of the return relative to an expected power spectrum, and/or determining that a duration associated with the signal portion meets or exceeds a threshold duration.

Abstract: A machine-learned (ML) model for detecting that depth data (e.g., lidar data, radar data) comprises a false positive attributable to particulate matter, such as dust, steam, smoke, rain, etc. The ML model may be trained based at least in part on simulated depth data generated by a fluid dynamics model and/or by collecting depth data during operation of a device (e.g., an autonomous vehicle. In some examples, an autonomous vehicle may identify depth data that may be associated with particulate matter based at least in part on an outlier region in a thermal image. For example, the outlier region may be associated with steam.

Abstract: A time delay of arrival (TDOA) between a time that a light pulse was emitted to a time that a pulse reflected off an object was received at a light sensor may be determined for saturated signals by using an edge of the saturated signal, rather than a peak of the signal, for the TDOA calculation. The edge of the saturated signal may be accurately estimated by fitting a first polynomial curve to data points of the saturated signal, defining an intermediate magnitude threshold based on the polynomial curve, fitting a second polynomial curve to data points near an intersection of the first polynomial curve and the intermediate threshold, and identifying an intersection of the second polynomial curve and the intermediate threshold as the rising edge of the saturated signal.

Abstract: A LIDAR system that identifies, from a channel output, a false positive return and/or suppressing a corresponding false positive detection caused, in some cases, a strong reflection by a highly reflective surface that caused light to leak from a first channel to a second channel. The LIDAR system described herein may identify, as a false return, a return detected in the second channel that has an intensity that is much less than a return in the first channel and indicates a distance that is the same or very close to a distance indicated the return in the first channel. Based at least in part on identifying a return as a false return, the LIDAR system may suppress a false detection associated with the false return by modifying a detection threshold.

Abstract: Sensors, including time-of-flight sensors, may be used to detect objects in an environment. In an example, a vehicle may include a time-of-flight sensor that images objects around the vehicle, e.g., so the vehicle can navigate relative to the objects. The sensor may generate first image data at a first configuration and second image data at a second configuration. The first image data and the second image data may be combined to provide disambiguated depth and improved intensity values for imaging the environment. In some examples, the first and second configurations may have different modulation frequencies, different integration times, and/or different illumination intensities. In some examples, configurations may be dynamically altered based on depth and/or intensity information of a previous frame.

Abstract: A vehicle computing system may implement techniques to dynamically adjust a volume and/or frequency of a sound emitted from a vehicle to warn an object (e.g., dynamic object) of a potential conflict with the vehicle. The techniques may include determining a baseline noise level and/or frequencies proximate to the object. The baseline noise level and/or frequencies may be determined based on an identification of one or more noise generating objects in the environment. The vehicle computing system may determine the volume and/or a frequency of the sound based in part on the baseline noise level and/or frequencies, an urgency of the warning, a probability of conflict between the vehicle and the object, a speed of the object, etc.

Abstract: Techniques for utilizing three-dimensional (3D) sound for passenger notification are described herein. Computing device(s) onboard a vehicle can determine an occurrence of an event associated with a passenger of the vehicle or the vehicle, and can determine a 3D sound associated with the event. Then, the computing device(s) can send a signal associated with the 3D sound to a speaker system inside of the vehicle and, responsive to receiving the signal, one or more speakers of the speaker system can output the 3D sound such that a sound associated with the 3D sound is perceived to be localized relative to the passenger in the vehicle.

Abstract: Techniques for using beam-formed acoustic notifications for pedestrian notification are described. Computing device(s) can receive sensor data associated with an object in an environment of a vehicle. The computing device(s) can determine first data for emitting a first beam of acoustic energy via speaker(s) of an acoustic array associated with the vehicle, and second data for emitting a second beam of acoustic energy via speakers of the acoustic array. The computing device(s) can cause the speaker(s) to emit the first beam in a direction of the object at a first time and the second beam in the direction of the object at a second time. Directions of propagation of the first beam and the second beam are offset so that the object can localize the source the acoustic notification, thereby localizing the vehicle in the environment.