Abstract: An embodiment according to the present invention includes a method for a machine-learning based approach to the formation of ultrasound and photoacoustic images. The machine-learning approach is used to reduce or remove artifacts to create a new type of high-contrast, high-resolution, artifact-free image. The method of the present invention uses convolutional neural networks (CNNs) to determine target locations to replace the geometry-based beamforming that is currently used. The approach is extendable to any application where beamforming is required, such as radar or seismography.

Abstract: Method of performing acoustic zooming starts with microphones capturing acoustic signals associated with video content. Beamformers generate beamformer signals using the acoustic signals. Beamformer signals correspond respectively to tiles of video content. Each of the beamformers is respectively directed to a center of each of the tiles. Target enhanced signal is generated using beamformer signals. Target enhanced signal is associated with a zoom area of video content. Target enhanced signal is generated by identifying the tiles respectively having at least portions that are included in the zoom area, selecting beamformer signals corresponding to identified tiles, and combining selected beamformer signals to generate target enhanced signal. Combining selected beamformer signals may include determining proportions for each of the identified tiles in relation to the zoom area and combining selected beamformer signals based on the proportions to generate the target enhanced signal.

Abstract: An embodiment in accordance with the present invention includes a technology to continuously measure patient mobility automatically, using sensors that capture color and depth images along with algorithms that process the data and analyze the activities of the patients and providers to assess the highest level of mobility of the patient. An algorithm according to the present invention employs the following five steps: 1) analyze individual images to locate the regions containing every person in the scene (Person Localization), 2) for each person region, assign an identity to distinguish ‘patient’ vs. ‘not patient’ (Patient Identification), 3) determine the pose of the patient, with the help of contextual information (Patient Pose Classification and Context Detection), 4) measure the degree of motion of the patient (Motion Analysis), and 5) infer the highest mobility level of the patient using the combination of pose and motion characteristics (Mobility Classification).

Abstract: Method of performing acoustic zooming starts with microphones capturing acoustic signals associated with video content. Beamformers generate beamformer signals using the acoustic signals. Beamformer signals correspond respectively to tiles of video content. Each of the beamformers is respectively directed to a center of each of the tiles. Target enhanced signal is generated using beamformer signals. Target enhanced signal is associated with a zoom area of video content. Target enhanced signal is generated by identifying the tiles respectively having at least portions that are included in the zoom area, selecting beamformer signals corresponding to identified tiles, and combining selected beamformer signals to generate target enhanced signal. Combining selected beamformer signals may include determining proportions for each of the identified tiles in relation to the zoom area and combining selected beamformer signals based on the proportions to generate the target enhanced signal.

Abstract: Method of performing acoustic zooming starts with microphones capturing acoustic signals associated with video content. Beamformers generate beamformer signals using the acoustic signals. Beamformer signals correspond respectively to tiles of video content. Each of the beamformers is respectively directed to a center of each of the tiles. Target enhanced signal is generated using beamformer signals. Target enhanced signal is associated with a zoom area of video content. Target enhanced signal is generated by identifying the tiles respectively having at least portions that are included in the zoom area, selecting beamformer signals corresponding to identified tiles, and combining selected beamformer signals to generate target enhanced signal. Combining selected beamformer signals may include determining proportions for each of the identified tiles in relation to the zoom area and combining selected beamformer signals based on the proportions to generate the target enhanced signal.

Abstract: Method of performing acoustic zooming starts with microphones capturing acoustic signals associated with video content. Beamformers generate beamformer signals using the acoustic signals. Beamformer signals correspond respectively to tiles of video content. Each of the beamformers is respectively directed to a center of each of the tiles. Target enhanced signal is generated using beamformer signals. Target enhanced signal is associated with a zoom area of video content. Target enhanced signal is generated by identifying the tiles respectively having at least portions that are included in the zoom area, selecting beamformer signals corresponding to identified tiles, and combining selected beamformer signals to generate target enhanced signal. Combining selected beamformer signals may include determining proportions for each of the identified tiles in relation to the zoom area and combining selected beamformer signals based on the proportions to generate the target enhanced signal.

Abstract: Method of performing acoustic zooming starts with microphones capturing acoustic signals associated with video content. Beamformers generate beamformer signals using the acoustic signals. Beamformer signals correspond respectively to tiles of video content. Each of the beamformers is respectively directed to a center of each of the tiles. Target enhanced signal is generated using beamformer signals. Target enhanced signal is associated with a zoom area of video content. Target enhanced signal is generated by identifying the tiles respectively having at least portions that are included in the zoom area, selecting beamformer signals corresponding to identified tiles, and combining selected beamformer signals to generate target enhanced signal. Combining selected beamformer signals may include determining proportions for each of the identified tiles in relation to the zoom area and combining selected beamformer signals based on the proportions to generate the target enhanced signal.

Abstract: A system for stereo reconstruction from a monoscopic endoscope includes an image pick-up element at a distal end thereof and a working channel defined by a body of the monoscopic endoscope. The system comprises a light patterning component configured to be disposed within the working channel such that a light emitting end of the light patterning component will be fixed with a defined relative distance from the distal end of the image pick-up element. The system also includes a data processor adapted to be in communication with the image pick-up element. The light patterning component forms a pattern of light that is projected onto a region of interest. The data processor receives an image signal of the region of interest that includes the pattern, and determines a distance from the endoscope to the region of interest based on the image signal and based on the defined relative distance.

Abstract: An embodiment in accordance with the present invention includes a technology to continuously measure patient mobility automatically, using sensors that capture color and depth images along with algorithms that process the data and analyze the activities of the patients and providers to assess the highest level of mobility of the patient. An algorithm according to the present invention employs the following five steps: 1) analyze individual images to locate the regions containing every person in the scene (Person Localization), 2) for each person region, assign an identity to distinguish ‘patient’ vs. ‘not patient’ (Patient Identification), 3) determine the pose of the patient, with the help of contextual information (Patient Pose Classification and Context Detection), 4) measure the degree of motion of the patient (Motion Analysis), and 5) infer the highest mobility level of the patient using the combination of pose and motion characteristics (Mobility Classification).

Abstract: An embodiment according to the present invention includes a method for a machine-learning based approach to the formation of ultrasound and photoacoustic images. The machine-learning approach is used to reduce or remove artifacts to create a new type of high-contrast, high-resolution, artifact-free image. The method of the present invention uses convolutional neural networks (CNNs) to determine target locations to replace the geometry-based beamforming that is currently used. The approach is extendable to any application where beamforming is required, such as radar or seismography.

Abstract: A system and method for tracking a surgical implement in a patient can have an imaging system configured to obtain sequential images of the patient, and an image recognition system coupled to the imaging system and configured to identify the surgical implement in individual images. The image recognition system can be configured to identify the surgical implement relative to the patient in one of the images based, at least in part, on an identification of the surgical implement in at least one preceding one of the sequential images, and a probabilistic analysis of individual sections of the one of the images, the sections being selected by the image recognition system based on a position of the surgical implement in the patient as identified in the at least one preceding one of the images.

Abstract: According to some embodiments of the present invention, a system for stereo reconstruction from a monoscopic endoscope is provided. The monoscopic endoscope comprising an image pick-up element at a distal end thereof and a working channel defined by a body of the monoscopic endoscope. The working channel provides a port at the distal end of the monoscopic endoscope. The system for stereo reconstruction comprises a light patterning component configured to be disposed within the working channel of the monoscopic endoscope such that a light emitting end of the light patterning component will be fixed with a defined relative distance from the distal end of the image pick-up element. The system for stereo reconstruction also includes a data processor adapted to be in communication with the image pick-up element. The light patterning component forms a pattern of light that is projected onto a region of interest.

Abstract: Appearance learning systems, methods and computer products for three-dimensional markerless tracking of robotic surgical tools. An appearance learning approach is provided that is used to detect and track surgical robotic tools in laparoscopic sequences. By training a robust visual feature descriptor on low-level landmark features, a framework is built for fusing robot kinematics and 3D visual observations to track surgical tools over long periods of time across various types of environments. Three-dimensional tracking is enabled on multiple tools of multiple types with different overall appearances. The presently disclosed subject matter is applicable to surgical robot systems such as the da Vinci® surgical robot in both ex vivo and in vivo environments.

Abstract: A system and method for tracking a surgical implement in a patient can have an imaging system configured to obtain sequential images of the patient, and an image recognition system coupled to the imaging system and configured to identify the surgical implement in individual images. The image recognition system can be configured to identify the surgical implement relative to the patient in one of the images based, at least in part, on an identification of the surgical implement in at least one preceding one of the sequential images, and a probabilistic analysis of individual sections of the one of the images, the sections being selected by the image recognition system based on a position of the surgical implement in the patient as identified in the at least one preceding one of the images.

Abstract: A Surgical Structured Light (SSL) system is disclosed that provides real-time, dynamic 3D visual information of the surgical environment, allowing registration of pre- and intra-operative imaging, online metric measurements of tissue, and improved navigation and safety within the surgical field.

Abstract: A surgical instrument has a distal end portion with an outer surface with an outer radius. One or more image capture elements are movably mounted in the distal end portion. In a first state, the one or more image capture elements are un-deployed. In the first state, a surface having an aperture of at least one of the one or more image capture elements is enclosed within the outer surface of the surgical instrument so that the surface having the aperture does not extend beyond the outer surface. In a second state, the one or more image capture elements are deployed. In the second state the surface having the aperture of the at least one of the one or more image capture elements extends beyond the outer surface.