Abstract: A method of detecting and responding to a visitor to a smart home environment via an electronic greeting system of the smart home environment, including determining that a visitor is approaching an entryway of the smart home environment; initiating a facial recognition operation while the visitor is approaching the entryway; initiating an observation window in response to the determination that a visitor is approaching the entryway; obtaining context information from one or more sensors of the smart home environment during the observation window; and at the end of the time window, initiating a response to the detected approach of the visitor based on the context information and/or an outcome of the facial recognition operation.

Abstract: Various arrangements are detailed herein related to managing video recording. A system can include a network-enabled video camera that captures a video stream of a location and streams the video stream to a cloud-based home assistant integration platform. The system can include a home assistant device comprising a microphone and wireless network interface, the home assistant device configured to receive, via the microphone, a spoken command that requests video, captured by the network-enabled video camera, be recorded. The cloud-based home assistant integration platform may be configured to analyze the video stream captured using the network-enabled video camera to determine an identity of the user. The platform may be further configured to store a portion of the video stream linked to a user account of the user based on the spoken command and the identity of the user.

Abstract: Systems and methods are described for improving endpoint detection of a voice query submitted by a user. In some implementations, a synchronized video data and audio data is received. A sequence of frames of the video data that includes images corresponding to lip movement on a face is determined. The audio data is endpointed based on first audio data that corresponds to a first frame of the sequence of frames and second audio data that corresponds to a last frame of the sequence of frames. A transcription of the endpointed audio data is generated by an automated speech recognizer. The generated transcription is then provided for output.

Abstract: Various arrangements are detailed herein related to managing video recording. A system can include a network-enabled video camera that captures a video stream of a location and streams the video stream to a cloud-based home assistant integration platform. The system can include a home assistant device comprising a microphone and wireless network interface, the home assistant device configured to receive, via the microphone, a spoken command that requests video, captured by the network-enabled video camera, be recorded. The cloud-based home assistant integration platform may be configured to analyze the video stream captured using the network-enabled video camera to determine an identity of the user. The platform may be further configured to store a portion of the video stream linked to a user account of the user based on the spoken command and the identity of the user.

Abstract: A method of detecting and responding to a visitor to a smart home environment via an electronic greeting system of the smart home environment, including determining that a visitor is approaching an entryway of the smart home environment; initiating a facial recognition operation while the visitor is approaching the entryway; initiating an observation window in response to the determination that a visitor is approaching the entryway; obtaining context information from one or more sensors of the smart home environment during the observation window; and at the end of the time window, initiating a response to the detected approach of the visitor based on the context information and/or an outcome of the facial recognition operation.

Abstract: A method of presenting appropriate actions for responding to a visitor to a smart home environment via an electronic greeting system of the smart home environment, including detecting a visitor of the smart home environment; obtaining context information from the smart home environment regarding the visitor; based on the context information, identifying a plurality of appropriate actions available to a user of a client device for interacting with the visitor via the electronic greeting system; and causing the identified actions to be presented to the user of the client device.

Abstract: The various embodiments described herein include methods, devices, and systems for ultrasonic sensing on electronic devices. In one aspect, a method is performed at an electronic device having memory, one or more processors, a speaker, and a microphone. The method includes, while audibly communicating with a user via the speaker and microphone: (1) sending one or more ultrasound pulses via the speaker; (2) receiving, via the microphone, one or more signals corresponding to the one or more ultrasound pulses; (3) determining positioning of the user based on the one or more received signals; and (4) adjusting one or more parameters of the speaker and/or the microphone based on the determined positioning.

Abstract: An electronic device has memory, one or more processors, a speaker, and a microphone. The device sends a first set of ultrasound chirps at a first rate via the speaker. It receives, via the microphone, a first set of signals corresponding to the first set of ultrasound chirps and being reflected from a person. The device determines based on the first set of signals that the person is in proximity to the electronic device. In accordance with the determination that the person is in proximity to the electronic device, the device sends a second set of ultrasound chirps at a second rate, faster than the first rate. It receives, via the microphone, a second set of signals corresponding to the second set of ultrasound chirps, and identifies a gesture from the person based on the second set of signals.

Abstract: Systems and methods are described for improving endpoint detection of a voice query submitted by a user. In some implementations, a synchronized video data and audio data is received. A sequence of frames of the video data that includes images corresponding to lip movement on a face is determined. The audio data is endpointed based on first audio data that corresponds to a first frame of the sequence of frames and second audio data that corresponds to a last frame of the sequence of frames. A transcription of the endpointed audio data is generated by an automated speech recognizer. The generated transcription is then provided for output.

Abstract: A method of detecting and responding to a visitor to a smart home environment via an electronic greeting system of the smart home environment, including determining that a visitor is approaching an entryway of the smart home environment; initiating a facial recognition operation while the visitor is approaching the entryway; initiating an observation window in response to the determination that a visitor is approaching the entryway; obtaining context information from one or more sensors of the smart home environment during the observation window; and at the end of the time window, initiating a response to the detected approach of the visitor based on the context information and/or an outcome of the facial recognition operation.

Abstract: Various arrangements are detailed herein related to managing video recording. A system can include a network-enabled video camera that captures a video stream of a location and streams the video stream to a cloud-based home assistant integration platform. The system can include a home assistant device comprising a microphone and wireless network interface, the home assistant device configured to receive, via the microphone, a spoken command that requests video, captured by the network-enabled video camera, be recorded. The cloud-based home assistant integration platform may be configured to analyze the video stream captured using the network-enabled video camera to determine an identity of the user. The platform may be further configured to store a portion of the video stream linked to a user account of the user based on the spoken command and the identity of the user.

Abstract: Various arrangements are detailed herein related to modifying a shopping list using a home assistant device and an integrated video service. A network-enabled video camera may be used to capture a video stream of a location at which a user is present. A home assistant device may receive a spoken command that requests an item be added to a shopping list. The video stream captured using the network-enabled video camera may be analyzed to determine an identity of the user. A shopping list to modify may be determined based on the identity of the user. The determined shopping list may then be modified to indicate the item requested in the spoken command.

Abstract: A method of detecting and responding to a visitor to a smart home environment via an electronic greeting system of the smart home environment, including determining that a visitor is approaching an entryway of the smart home environment; initiating a facial recognition operation while the visitor is approaching the entryway; initiating an observation window in response to the determination that a visitor is approaching the entryway; obtaining context information from one or more sensors of the smart home environment during the observation window; and at the end of the time window, initiating a response to the detected approach of the visitor based on the context information and/or an outcome of the facial recognition operation.

Abstract: Various arrangements are detailed herein related to modifying a shopping list using a home assistant device and an integrated video service. A network-enabled video camera may be used to capture a video stream of a location at which a user is present. A home assistant device may receive a spoken command that requests an item be added to a shopping list. The video stream captured using the network-enabled video camera may be analyzed to determine an identity of the user. A shopping list to modify may be determined based on the identity of the user. The determined shopping list may then be modified to indicate the item requested in the spoken command.

Abstract: A method of presenting appropriate actions for responding to a visitor to a smart home environment via an electronic greeting system of the smart home environment, including detecting a visitor of the smart home environment; obtaining context information from the smart home environment regarding the visitor; based on the context information, identifying a plurality of appropriate actions available to a user of a client device for interacting with the visitor via the electronic greeting system; and causing the identified actions to be presented to the user of the client device.

Abstract: A method of detecting and responding to a visitor to a smart home environment via an electronic greeting system of the smart home environment, including determining that a visitor is approaching an entryway of the smart home environment; initiating a facial recognition operation while the visitor is approaching the entryway; initiating an observation window in response to the determination that a visitor is approaching the entryway; obtaining context information from one or more sensors of the smart home environment during the observation window; and at the end of the time window, initiating a response to the detected approach of the visitor based on the context information and/or an outcome of the facial recognition operation.

Abstract: Various arrangements are detailed herein related to managing scheduled events using a home assistant and an integrated video service. Using a network-enabled video camera, a video stream of a location may be captured. The video stream captured may be used to determine an identity of a user present at the location. In response to identifying the identity of the user, access to a cloud-based calendar account for the user may be determined to be authorized. The cloud-based calendar account linked with the user may be accessed. A next calendar event mapped to the cloud-based calendar account that is scheduled to occur within a threshold time duration may be determined. A description linked with the next calendar event may be accessed. A network-enabled smart home assistant device may be used to synthesize speech to announce details related to the next event.

Abstract: Various arrangements are detailed herein related to managing scheduled events using a home assistant and an integrated video service. Using a network-enabled video camera, a video stream of a location may be captured. The video stream captured may be used to determine an identity of a user present at the location. In response to identifying the identity of the user, access to a cloud-based calendar account for the user may be determined to be authorized. The cloud-based calendar account linked with the user may be accessed. A next calendar event mapped to the cloud-based calendar account that is scheduled to occur within a threshold time duration may be determined. A description linked with the next calendar event may be accessed. A network-enabled smart home assistant device may be used to synthesize speech to announce details related to the next event.

Abstract: Background noise is modeled from an input signal comprising a desired signal and a plurality of undesired signals. At least one of the signals that comprise the input is processed to generate a signal-to-noise ratio. An articulation index is generated for the at least one of the signals that is processed. A spectrum of a speech segment is generated to improve intelligibility and quality of the speech segment based on the articulation index. A shaping logic may adjust the spectrum of the speech segment based on a comparison of the articulation index to a plurality of predetermined thresholds. Modeling of the background noise comprises modeling a tilt of the background noise.

Abstract: A system improves speech intelligibility by reconstructing speech segments. The system includes a low-frequency reconstruction controller programmed to select a predetermined portion of a time domain signal. The low-frequency reconstruction controller substantially blocks signals above and below the selected predetermined portion. A harmonic generator generates low-frequency harmonics in the time domain that lie within a frequency range controlled by a background noise modeler. A gain controller adjusts the low-frequency harmonics to substantially match the signal strength to the time domain original input signal.