Abstract: A method for responding to a user query based on captured images and audio. An audio signal captured by at least one microphone is analyzed to determine at least one word. At least one image captured by at least one image sensor is analyzed to determine at least one identifier of at least one of a person, an object, a location, or an event represented in the image. The at least one word and the at least one identifier are stored in a database. A question is received from the user and is analyzed to determine at least one term. The database is searched to determine a correlation between the at least one term and the at least one word or between the at least one term and the at least one identifier. A response to the question is generated based on the correlation and is provided to the user.

Abstract: A wearable device may include an image sensor configured to capture a plurality of images from an environment, a microphone configured to capture sounds from the environment, and at least one processor. The at least one processor may be programmed to receive audio signals representative of the sounds captured by the at least one microphone, and receive a first image including a representation of a first individual from among the plurality of images captured by the image sensor. The at least one processor may also be programmed to obtain a first audio segment from the audio signals using the first image. The first audio segment may include a first portion of the audio signals in which the first individual is speaking. The at least one processor may also be programmed to receive a second image including a representation of a second individual from among the plurality of images captured by the image sensor, and obtain a second audio segment from the audio signals using the second image.

Abstract: A hearing aid system for generating processed audio signals may include at least one processor. The processor may be configured to receive an audio signal representative of the sounds captured by the at least one microphone; generate a processed audio signal based on analysis of the audio signal; determine a quality degradation of at least one aspect of the processed audio signal; and provide an auditory cue to the user indicating the quality degradation.

Abstract: A wearable device may include a housing, a sensor in the housing configured to generate an output, and a transmitter in the housing. The wearable device may also include a processor programmed to alternatively operate in a normal radiation mode and a low radiation mode. The transmitter may be permitted to function at a normal capacity when operating in the normal radiation mode and may be caused to function at a reduced capacity when operating in the low radiation mode. During operation at the normal capacity, the transmitter may transmit at a higher radiation intensity than during operation at the reduced capacity. The processor may also be programmed to detect, based on the output generated by the sensor, whether the housing is currently worn by the user, and cause the transmitter to operate in the low radiation mode after detecting that the housing is being worn by the user.

Abstract: Systems and methods are disclosed for locating an object for a user. A system may comprise an image capture device, an audio capture device, and a processor. The processor may be configured to receive images captured by the image capture device and audio signals received by the audio capture device. The processor may analyze the audio signals to identify a descriptor word describing the object and retrieve a visual characteristic of the object based on the descriptor word. The processor may then determine a location of the object in the images based on the visual characteristic, determine a location of a hand of the user in the images, and determine a direction between the hand and the object. The processor may then determine feedback indicative of the direction and provide the feedback to the user.

Abstract: A system may include a wearable camera configured to capture a plurality of images from an environment of a user and a microphone configured to capture sounds from an environment of the user. The system may also include a processor programmed to receive the images; identify a representation of an individual in one of the images; identify a lip movement associated with a mouth of the individual, based on analysis of the images; receive audio signals representative of the sounds; identify, based on analysis of the sounds, an audio signal associated with a voice of the individual; and cause transmission of the audio signal to a hearing interface device configured to provide sound to an ear of the user.

Abstract: A navigation system for a host vehicle is provided. The system may comprise at least one processing device comprising circuitry and a memory. The memory includes instructions that when executed by the circuitry cause the at least one processing device to: receive a plurality of images acquired by a camera, the plurality of images being representative of an environment of the host vehicle; analyze the plurality of images to identify a presence in the environment of the host vehicle a navigation rule suspension condition; temporarily suspend at least one navigational rule in response to identification of the navigation rule suspension condition; and cause at least one navigational change of the host vehicle unconstrained by the temporarily suspended at least one navigational rule.

Abstract: A wearable apparatus and methods may analyze images. In one implementation, a wearable apparatus for capturing and processing images may comprise a wearable image sensor configured to capture a plurality of images from an environment of a user of the wearable apparatus and at least one programming device. The at least one processing device may be programmed to: perform a first analysis of the plurality of images to detect at least two persons; perform a second analysis of the plurality of images to determine association information related to the at least two detected persons; and update a social representation based on the determined association information.

Abstract: A wearable apparatus is provided. The wearable apparatus may include: a wearable image sensor configured to capture a plurality of images from an environment of a user; and at least one processor programmed to: receive, from the wearable image sensor, a facial image of an individual with whom a user interacted in a first interaction during a time window; receive sound data captured in a vicinity of the image sensor during a part of the time window; process the sound data to identify a key word; store an association between the key word and the facial image; receive another facial image of the individual during a second interaction; determine that the individual is the individual in the second interaction; access the memory to locate the key word from the first interaction; and cause a display of the key word on a display visible to the user.

Abstract: System and methods for processing audio signals are disclosed. In one implementation, a system may include a wearable apparatus including an image sensor to capture images from an environment of a user; an audio sensor to capture an audio signal from the environment of the user; and at least one processor. The processor may be programmed to receive the audio signal captured by the audio sensor; identify at least one segment including speech in the audio signal; receive an image including a representation of a code; analyze the code to determine whether the code is associated with the user and/or the wearable apparatus; and after determining that the code is associated with the user and/or the wearable apparatus, transmit at least one segment of the audio signal, at least one image of the plurality of images, and/or other information to a computing platform.

Abstract: A system may include a camera configured to capture images from an environment of a user and a microphone configured to capture sounds from an environment of the user. The system may also include a processor programmed to: receive the images; identify a representation of a first individual and a representation of a second individual in the images; receive, from the microphone, a first audio signal associated with a voice of the first individual and a second audio signal associated with a voice of the second individual; detect an amplification criteria indicative of a voice amplification priority between the first individual and the second individual; selectively amplify the first audio signal relative to the second audio signal when the amplification criteria indicates that the first individual has voice amplification priority over the second individual; and cause transmission of the selectively amplified first audio signal to a hearing interface device.

Abstract: A system may include a wearable camera configured to capture a plurality of images from an environment of a user and a microphone configured to capture sounds from an environment of the user. The system may also include a processor programmed to receive the images; identify a representation of one individual in one of the images; identify a lip movement associated with a mouth of the individual, based on analysis of the images; receive audio signals representative of the sounds; identify, based on analysis of the sounds, a first audio signal associated with a first voice and a second audio signal associated with a second voice; cause selective conditioning of the first audio signal based on a determination that the first audio signal is associated with the identified lip movement; and cause transmission of the selectively conditioned first audio signal to a hearing interface device.

Abstract: Systems and methods use cameras to provide autonomous navigation features. In one implementation, a driver-assist object detection system is provided for a vehicle. One or more processing devices associated with the system receive at least two images from a plurality of captured images via a data interface. The device(s) analyze the first image and at least a second image to determine a reference plane corresponding to the roadway the vehicle is traveling on. The processing device(s) locate a target object in the first two images, and determine a difference in a size of at least one dimension of the target object between the two images. The system may use the difference in size to determine a height of the object. Further, the system may cause a change in at least a directional course of the vehicle if the determined height exceeds a predetermined threshold.

Abstract: A wearable apparatus may automatically monitor consumption by a user of the wearable apparatus by analyzing images captured from an environment of the user. The wearable apparatus may include at least one image capture device configured to capture a plurality of images from an environment of the user of the wearable apparatus.

Abstract: A hearing aid and related systems and methods are disclosed. In one implementation, a hearing aid system may include a wearable camera; a microphone; and a processor. The processor may be programmed to receive images captured by the camera; receive audio signals representative of sounds received by the at least one microphone; determine a look direction of the user based on analysis of the images; determine an amplitude of a first audio signal associated with an individual or object in a region associated with the look direction of the user; determine an amplitude of a second audio signal from a region other than the look direction of the user; adjust the second amplitude in accordance with the first amplitude; and cause transmission of the second audio signal at the adjusted amplitude to a hearing interface device configured to provide sound to an ear of the user.

Abstract: Systems and methods are disclosed for using a wearable apparatus with overhead projection. In one implementation, a wearable apparatus may include an image sensor configured to capture a plurality of images from an environment of a user; a light projector; and at least one processor. The processor may be programmed to receive at least one image of the plurality of images; identify an item of information based on the at least one image; select a surface located within a field of view of the user; render the item of information into an informational image; and cause the projector to project the informational image onto the surface.

Abstract: A hearing aid and related systems and methods are disclosed. In one implementation, a system may comprise a microphone, a wearable camera, and a processor. The processor may be configured to receive a composite audio signal representative of sounds captured by the microphone, the composite audio signal including a representation of an audio source and additional audio source in the environment of the user; obtain an indication of a direction of arrival associated with the audio source, the direction of arrival representing a position of the audio source relative to the user; provide the composite audio signal, information associated with the plurality of images, and the indication of the direction of arrival to a trained model; and extract, based on an output from the trained model, an isolated audio signal from the composite audio signal, the isolated audio signal representing sounds emanating from the audio source.

Abstract: A method including operations to obtain a planned driving action for accomplishing a navigational goal of a host vehicle on a roadway, identify a planned trajectory for the host vehicle, corresponding to the planned driving action, identify, from sensor data representative of an environment of the host vehicle, an occluded location of a potential object that is occluded from view of the host vehicle, identify a possible trajectory of the potential object, based on possible movement of the potential object from the location into the roadway, identify an intersection of the planned trajectory for the host vehicle with the possible trajectory for the potential object, determine a safety action of the host vehicle to respond to the possible movement of the potential object, and apply the safety action to change the planned driving action of the host vehicle.

Abstract: System and methods for selectively amplifying audio signals are disclosed. In one implementation, a method includes receiving at least one image of a plurality of images captured by a wearable camera; receiving a first audio signal representative of the sounds captured by a microphone; determining a looking direction of a user; and processing the first audio signal by amplifying audio coming from the looking direction of the user and attenuating audio coming from at least one other direction; receiving a second audio signal representative of the sounds captured by a hearing interface device; transmitting the second audio signal to a speaker associated with the hearing interface device; transmitting an additional audio signal to the speaker, wherein the transmission of the additional audio signal at least partially overlaps the transmission of the second audio signal; and transmitting the processed first audio signal to the speaker.

Abstract: Systems and methods are provided for vehicle navigation. In one implementation, a system may comprise an interface to obtain sensing data of an environment of the host vehicle. The processing device may be configured to determine a planned navigational action; identify, a target vehicle in the environment of the host vehicle; predict a distance between the host vehicle and the target vehicle if the planned navigational action was taken; determine a current host vehicle stopping distance based on a braking capability, acceleration capability, and speed of the host vehicle; determine a current target vehicle braking distance based on a speed and braking capability of the target vehicle; and implement the planned navigational action when the predicted distance of the planned navigational action is greater than a minimum safe longitudinal distance calculated based on the current host vehicle stopping distance and the current target vehicle braking distance.