Abstract: One or more machine learning techniques can be used to identify locations of potential malignancies within images (e.g., video images) captured during a medical procedure, such as an endoscopic procedure. The images can be displayed, in real-time, on a display unit. The images can be displayed with a graphical overlay that isolates the identified locations of potential malignancies.

Abstract: Remote automated assistant component(s) generate client device notification(s) based on a received IoT state change notification that indicates a change in at least one state associated with at least one IoT device. The generated client device notification(s) can each indicate the change in state associated with the at least one IoT device, and can optionally indicate the at least one IoT device. Further, the remote automated assistant component(s) can identify candidate assistant client devices that are associated with the at least one IoT device, and determine whether each of the one or more of the candidate assistant client device(s) should render a corresponding client device notification.

Abstract: Various arrangements are detailed herein related to managing video recording. A system can include a network-enabled video camera that captures a video stream of a location and streams the video stream to a cloud-based home assistant integration platform. The system can include a home assistant device comprising a microphone and wireless network interface, the home assistant device configured to receive, via the microphone, a spoken command that requests video, captured by the network-enabled video camera, be recorded. The cloud-based home assistant integration platform may be configured to analyze the video stream captured using the network-enabled video camera to determine an identity of the user. The platform may be further configured to store a portion of the video stream linked to a user account of the user based on the spoken command and the identity of the user.

Abstract: Methods, devices, systems, and means for video integration with a home assistant device are described herein. The home assistant device interacts with a person in a video stream by capturing, using a network-enabled outdoor video camera, a video stream of an outdoor location of a premises at which the person is present and analyzing the person appearing in the captured video stream to determine an identity of the person. Based on determining the identity of the person, the home assistant device announces a presence of the person that is outdoors and outputs instructions to the person.

Abstract: Remote automated assistant component(s) generate client device notification(s) based on a received IoT state change notification that indicates a change in at least one state associated with at least one IoT device. The generated client device notification(s) can each indicate the change in state associated with the at least one IoT device, and can optionally indicate the at least one IoT device. Further, the remote automated assistant component(s) can identify candidate assistant client devices that are associated with the at least one IoT device, and determine whether each of the one or more of the candidate assistant client device(s) should render a corresponding client device notification.

Abstract: Generating a structured medical record from endoscopy data includes obtaining image data including endoscopic images representing portions of a gastrointestinal tract (GI) of a patient; determining features to extract from the image data, the features each representing a physical parameter of the GI tract; extracting the features from the image data; generating anatomical location data specifying a location within the GI tract of a portion of the GI tract represented in the image data; associating the anatomical location data with images that represent the portion of the GI tract; storing, in a node of a data store, data entries including the anatomical location data and the associated one or more images. The data store is configured to receive structured queries for the data entries in the data store and provide the data entries including the transformed features in response to receiving the structured queries.

Abstract: Systems and methods for tracking an evolution of a disease in a colon of a patient over time are configured for operations including receiving video data representing a colon of a patient; segmenting the video data into segments representing portions of a colon of the patient; extracting, from the video data based on the segmenting, a set of features representing locations in the colon of the patient; and registering the feature vector as representing the colon of the patient for tracking the evolution of the disease in the colon of the patient. The system can be configured to predict disease progression predict drug dosage for patients.

Abstract: A system is described that receives a query model of a query that includes one or more query commands. The query model includes a command model that corresponds to at least query command of the one or more query commands. The system uses the command model to generate an interactive action model summary and causes a user interface to display the query and the interactive action model summary in a query actions panel. A modification to the query in the user interface causes an update to the query actions panel and a modification to the action model summary causes an update to the at least one query command of the query.

Abstract: A system is configured to detect a small, but meaningful, anomaly within one or more metrics associated with a platform. The system displays visuals of the metrics so that a user monitoring the platform can effectively notice a problem associated with the anomaly and take appropriate action to remediate the problem. A first visual includes a radar-based visual that renders an object representing data for a set of metrics being monitored. A second visual includes a tree map visual that includes sections where each section is associated with an attribute used to compose the set of metrics. Via the display of the visuals, the techniques provide an improved way of representing a large number of metrics (e.g., hundreds, thousands, etc.) being monitored for a platform. Moreover, the techniques are configured to expose useful information associated with the platform in a manner that can be effectively interpreted by a user.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for predicting gaze and awareness using a neural network model. One of the methods includes obtaining sensor data (i) that is captured by one or more sensors of an autonomous vehicle and (ii) that characterizes an agent that is in a vicinity of the autonomous vehicle in an environment at a current time point. The sensor data is processed using a gaze prediction neural network to generate a gaze prediction that predicts a gaze of the agent at the current time point. The gaze prediction neural network includes an embedding subnetwork that is configured to process the sensor data to generate an embedding characterizing the agent, and a gaze subnetwork that is configured to process the embedding to generate the gaze prediction.

Abstract: Remote automated assistant component(s) generate client device notification(s) based on a received IoT state change notification that indicates a change in at least one state associated with at least one IoT device. The generated client device notification(s) can each indicate the change in state associated with the at least one IoT device, and can optionally indicate the at least one IoT device. Further, the remote automated assistant component(s) can identify candidate assistant client devices that are associated with the at least one IoT device, and determine whether each of the one or more of the candidate assistant client device(s) should render a corresponding client device notification.

Abstract: Aspects of the disclosure provide for the identification of parkable areas. In one instance, observations of parked vehicles may be identified from logged data. The observations may be used to determine whether a sub-portion of an edge of a roadgraph corresponds to a parkable area. In some examples, the edge may define a drivable area in the roadgraph. In addition, map information is generated based on the determination of whether the sub-portion of the edge corresponds to the parkable area.

Abstract: Various arrangements are detailed herein related to managing video recording. A system can include a network-enabled video camera that captures a video stream of a location and streams the video stream to a cloud-based home assistant integration platform. The system can include a home assistant device comprising a microphone and wireless network interface, the home assistant device configured to receive, via the microphone, a spoken command that requests video, captured by the network-enabled video camera, be recorded. The cloud-based home assistant integration platform may be configured to analyze the video stream captured using the network-enabled video camera to determine an identity of the user. The platform may be further configured to store a portion of the video stream linked to a user account of the user based on the spoken command and the identity of the user.

Abstract: A computerized method is disclosed. The method comprises detecting a change in a first webpage, constructing a search query based on the changed detected in the first webpage, determining a second webpage is associated with the first webpage, and synchronizing at least a portion of the second webpage with the first webpage based on the search query. The method may comprise updating a display of the first webpage based on the detected change. In certain embodiments of the method, detecting the change includes detecting input indicating one or more search parameters and the search query may be a text string including one or more alphanumeric characters. Additionally, the search query may comprise a pipelined command language. The synchronization of webpages displayed in separate tabs or windows of a web browser may use a browser extension to maintain state, as well as track and coordinate changes between the tabs or windows.

Abstract: Various arrangements are detailed herein related to managing video recording. A system can include a network-enabled video camera that captures a video stream of a location and streams the video stream to a cloud-based home assistant integration platform. The system can include a home assistant device comprising a microphone and wireless network interface, the home assistant device configured to receive, via the microphone, a spoken command that requests video, captured by the network-enabled video camera, be recorded. The cloud-based home assistant integration platform may be configured to analyze the video stream captured using the network-enabled video camera to determine an identity of the user. The platform may be further configured to store a portion of the video stream linked to a user account of the user based on the spoken command and the identity of the user.

Abstract: This specification describes systems and methods for obtaining various patient related data for inflammatory bowel disease (IBD). The methods and systems are configured for using machine learning to determine measurements of various characteristics and provide analysis related to IBD. The methods and systems may also obtain and incorporate electronic health data as well as other relevant data of patients along with endoscopic data to use for prediction IDB progression and recommending treatment.

Abstract: One or more machine learning techniques can be used to identify locations of potential malignancies within images (e.g., video images) captured during a medical procedure, such as an endoscopic procedure. The images can be displayed, in real-time, on a display unit.

Abstract: This specification describes systems and methods for performing endoscopy, obtaining medical images for inflammatory bowel disease (IBD) and scoring severity of IBD in patients. The methods and systems are configured for using machine learning to determine measurements of various characteristics related to IBD. The methods and systems may also obtain and incorporate electronic health data of patients along with endoscopic data to use for scoring purposes.

Abstract: A system is configured to detect a small, but meaningful, anomaly within one or more metrics associated with a platform. The system displays visuals of the metrics so that a user monitoring the platform can effectively notice a problem associated with the anomaly and take appropriate action to remediate the problem. A first visual includes a radar-based visual that renders an object representing data for a set of metrics being monitored. A second visual includes a tree map visual that includes sections where each section is associated with an attribute used to compose the set of metrics. Via the display of the visuals, the techniques provide an improved way of representing a large number of metrics (e.g., hundreds, thousands, etc.) being monitored for a platform. Moreover, the techniques are configured to expose useful information associated with the platform in a manner that can be effectively interpreted by a user.

Abstract: A system is configured to detect a small, but meaningful, anomaly within one or more metrics associated with a platform. The system displays visuals of the metrics so that a user monitoring the platform can effectively notice a problem associated with the anomaly and take appropriate action to remediate the problem. A first visual includes a radar-based visual that renders an object representing data for a set of metrics being monitored. A second visual includes a tree map visual that includes sections where each section is associated with an attribute used to compose the set of metrics. Via the display of the visuals, the techniques provide an improved way of representing a large number of metrics (e.g., hundreds, thousands, etc.) being monitored for a platform. Moreover, the techniques are configured to expose useful information associated with the platform in a manner that can be effectively interpreted by a user.