Patents by Inventor Neal Wadhwa
Neal Wadhwa has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 10636149Abstract: An apparatus according to an embodiment of the present invention enables measurement and visualization of a refractive field such as a fluid. An embodiment device obtains video captured by a video camera with an imaging plane. Representations of apparent motions in the video are correlated to determine actual motions of the refractive field. A textured background of the scene can be modeled as stationary, with a refractive field translating between background and video camera. This approach offers multiple advantages over conventional fluid flow visualization, including an ability to use ordinary video equipment outside a laboratory without particle injection. Even natural backgrounds can be used, and fluid motion can be distinguished from refraction changes. Embodiments can render refractive flow visualizations for augmented reality, wearable devices, and video microscopes.Type: GrantFiled: November 21, 2017Date of Patent: April 28, 2020Assignee: Massachusetts Institute of TechnologyInventors: William T. Freeman, Frederic Durand, Tianfan Xue, Michael Rubinstein, Neal Wadhwa
-
Patent number: 10380745Abstract: A method and corresponding apparatus for measuring object motion using camera images may include measuring a global optical flow field of a scene. The scene may include target and reference objects captured in an image sequence. Motion of a camera used to capture the image sequence may be determined relative to the scene by measuring an apparent, sub-pixel motion of the reference object with respect to an imaging plane of the camera. Motion of the target object corrected for the camera motion may be calculated based on the optical flow field of the scene and on the apparent, sub-pixel motion of the reference object with respect to the imaging plane of the camera. Embodiments may enable measuring vibration of structures and objects from long distance in relatively uncontrolled settings, with or without accelerometers, with high signal-to-noise ratios.Type: GrantFiled: February 28, 2017Date of Patent: August 13, 2019Assignee: Massachusetts Institute of TechnologyInventors: Oral Buyukozturk, William T. Freeman, Frederic Durand, Myers Abraham Davis, Neal Wadhwa, Justin G. Chen
-
Patent number: 10242427Abstract: Geometries of the structures and objects deviate from their idealized models, while not always visible to the naked eye. Embodiments of the present invention reveal and visualize such subtle geometric deviations, which can contain useful, surprising information. In an embodiment of the present invention, a method can include fitting a model of a geometry to an input image, matting a region of the input image according to the model based on a sampling function, generating a deviation function based on the matted region, extrapolating the deviation function to an image wide warping field, and generating an output image by warping the input image according to the warping. In an embodiment of the present invention, Deviation Magnification inputs takes a still image or frame, fits parametric models to objects of interest, and generates an output image exaggerating departures from ideal geometries.Type: GrantFiled: July 29, 2016Date of Patent: March 26, 2019Assignee: Massachusetts Institute of TechnologyInventors: Neal Wadhwa, Tali Dekel, Donglai Wei, Frederic Pierre Durand, William T. Freeman
-
Patent number: 10217218Abstract: In an embodiment, a method converts two images to a transform representation in a transform domain. For each spatial position, the method examines coefficients representing a neighborhood of the spatial position that is spatially the same across each of the two images. The method calculates a first vector in the transform domain based on first coefficients representing the spatial position, the first vector representing change from a first to second image of the two images describing deformation. The method modifies the first vector to create a second vector in the transform domain representing amplified movement at the spatial position between the first and second images. The method calculates second coefficients based on the second vector of the transform domain. From the second coefficients, the method generates an output image showing motion amplified according to the second vector for each spatial position between the first and second images.Type: GrantFiled: May 17, 2018Date of Patent: February 26, 2019Assignees: Massachusetts Institute of Technology, Quanta Computer Inc.Inventors: Hao-yu Wu, Michael Rubinstein, Eugene Inghaw Shih, John V. Guttag, Frederic Durand, William T. Freeman, Neal Wadhwa
-
Publication number: 20190035086Abstract: A method and corresponding apparatus for measuring object motion using camera images may include measuring a global optical flow field of a scene. The scene may include target and reference objects captured in an image sequence. Motion of a camera used to capture the image sequence may be determined relative to the scene by measuring an apparent, sub-pixel motion of the reference object with respect to an imaging plane of the camera. Motion of the target object corrected for the camera motion may be calculated based on the optical flow field of the scene and on the apparent, sub-pixel motion of the reference object with respect to the imaging plane of the camera. Embodiments may enable measuring vibration of structures and objects from long distance in relatively uncontrolled settings, with or without accelerometers, with high signal-to-noise ratios.Type: ApplicationFiled: February 28, 2017Publication date: January 31, 2019Inventors: Oral Buyukozturk, William T. Freeman, Frederic Durand, Myers Abraham Davis, Neal Wadhwa, Justin G. Chen
-
Patent number: 10129658Abstract: A method of recovering audio signals and corresponding apparatus according to an embodiment of the present invention using video or other sequence of images enables recovery of sound that causes vibrations of a surface. An embodiment method includes combining representations of local motions of a surface to produce a global motion signal of the surface. The local motions are captured in a series of images of features of the surface, and the global motion signal represents a sound within an environment in which the surface is located. Some embodiments compare representations of local motions of a surface to determine which motions are in-phase or out-of-phase with each other, enabling visualization of surface vibrational modes. Embodiments are passive, as compared to other forms of remote audio recovery that employ active sensing, such as laser microphone systems. Example applications for the embodiments include espionage and surveillance.Type: GrantFiled: July 21, 2014Date of Patent: November 13, 2018Assignee: Massachusetts Institute of TechnologyInventors: Michael Rubinstein, Myers Abraham Davis, Frederic Durand, William T. Freeman, Neal Wadhwa
-
Publication number: 20180268543Abstract: In an embodiment, a method converts two images to a transform representation in a transform domain. For each spatial position, the method examines coefficients representing a neighborhood of the spatial position that is spatially the same across each of the two images. The method calculates a first vector in the transform domain based on first coefficients representing the spatial position, the first vector representing change from a first to second image of the two images describing deformation. The method modifies the first vector to create a second vector in the transform domain representing amplified movement at the spatial position between the first and second images. The method calculates second coefficients based on the second vector of the transform domain. From the second coefficients, the method generates an output image showing motion amplified according to the second vector for each spatial position between the first and second images.Type: ApplicationFiled: May 17, 2018Publication date: September 20, 2018Inventors: Hao-yu Wu, Michael Rubinstein, Eugene Inghaw Shih, John V. Guttag, Frederic Durand, William T. Freeman, Neal Wadhwa
-
Patent number: 10037609Abstract: A method and corresponding device for identifying operational mode shapes of an object in a video stream includes extracting pixel-wise Eulerian motion signals of an object from an undercomplete representation of frames within a video stream. Pixel-wise Eulerian motion signals are downselected to produce a representative set of Eulerian motion signals of the object. Operational mode shapes of the object are identified based on the representative set. Resonant frequencies can also be identified. Embodiments enable vibrational characteristics of objects to be determined using video in near real time.Type: GrantFiled: February 1, 2016Date of Patent: July 31, 2018Assignee: Massachusetts Institute of TechnologyInventors: Justin Gejune Chen, Oral Buyukozturk, William T. Freeman, Frederic Pierre Durand, Myers Abraham Davis, Neal Wadhwa
-
Patent number: 10007986Abstract: In an embodiment, a method converts two images to a transform representation in a transform domain. For each spatial position, the method examines coefficients representing a neighborhood of the spatial position that is spatially the same across each of the two images. The method calculates a first vector in the transform domain based on first coefficients representing the spatial position, the first vector representing change from a first to second image of the two images describing deformation. The method modifies the first vector to create a second vector in the transform domain representing amplified movement at the spatial position between the first and second images. The method calculates second coefficients based on the second vector of the transform domain. From the second coefficients, the method generates an output image showing motion amplified according to the second vector for each spatial position between the first and second images.Type: GrantFiled: September 29, 2017Date of Patent: June 26, 2018Assignees: MASSACHUSETTS INSTITUTE OF TECHNOLOGY, QUANTA COMPUTER, INC.Inventors: Hao-yu Wu, Michael Rubinstein, Eugene Inghaw Shih, John V. Guttag, Frederic Durand, William T. Freeman, Neal Wadhwa
-
Publication number: 20180096482Abstract: An apparatus according to an embodiment of the present invention enables measurement and visualization of a refractive field such as a fluid. An embodiment device obtains video captured by a video camera with an imaging plane. Representations of apparent motions in the video are correlated to determine actual motions of the refractive field. A textured background of the scene can be modeled as stationary, with a refractive field translating between background and video camera. This approach offers multiple advantages over conventional fluid flow visualization, including an ability to use ordinary video equipment outside a laboratory without particle injection. Even natural backgrounds can be used, and fluid motion can be distinguished from refraction changes. Embodiments can render refractive flow visualizations for augmented reality, wearable devices, and video microscopes.Type: ApplicationFiled: November 21, 2017Publication date: April 5, 2018Inventors: William T. Freeman, Frederic Durand, Tianfan Xue, Michael Rubinstein, Neal Wadhwa
-
Publication number: 20180061063Abstract: A method and corresponding apparatus for measuring object motion using camera images may include measuring a global optical flow field of a scene. The scene may include target and reference objects captured in an image sequence. Motion of a camera used to capture the image sequence may be determined relative to the scene by measuring an apparent, sub-pixel motion of the reference object with respect to an imaging plane of the camera. Motion of the target object corrected for the camera motion may be calculated based on the optical flow field of the scene and on the apparent, sub-pixel motion of the reference object with respect to the imaging plane of the camera. Embodiments may enable measuring vibration of structures and objects from long distance in relatively uncontrolled settings, with or without accelerometers, with high signal-to-noise ratios.Type: ApplicationFiled: February 28, 2017Publication date: March 1, 2018Inventors: Oral Buyukozturk, William T. Freeman, Frederic Durand, Myers Abraham Davis, Neal Wadhwa, Justin G. Chen
-
Publication number: 20180047160Abstract: In an embodiment, a method converts two images to a transform representation in a transform domain. For each spatial position, the method examines coefficients representing a neighborhood of the spatial position that is spatially the same across each of the two images. The method calculates a first vector in the transform domain based on first coefficients representing the spatial position, the first vector representing change from a first to second image of the two images describing deformation. The method modifies the first vector to create a second vector in the transform domain representing amplified movement at the spatial position between the first and second images. The method calculates second coefficients based on the second vector of the transform domain. From the second coefficients, the method generates an output image showing motion amplified according to the second vector for each spatial position between the first and second images.Type: ApplicationFiled: September 29, 2017Publication date: February 15, 2018Applicant: Quanta Computer, Inc.Inventors: Hao-yu Wu, Michael Rubinstein, Eugene Inghaw Shih, John V. Guttag, Frederic Durand, William T. Freeman, Neal Wadhwa
-
Publication number: 20180032838Abstract: Geometries of the structures and objects deviate from their idealized models, while not always visible to the naked eye. Embodiments of the present invention reveal and visualize such subtle geometric deviations, which can contain useful, surprising information. In an embodiment of the present invention, a method can include fitting a model of a geometry to an input image, matting a region of the input image according to the model based on a sampling function, generating a deviation function based on the matted region, extrapolating the deviation function to an image wide warping field, and generating an output image by warping the input image according to the warping. In an embodiment of the present invention, Deviation Magnification inputs takes a still image or frame, fits parametric models to objects of interest, and generates an output image exaggerating departures from ideal geometries.Type: ApplicationFiled: July 29, 2016Publication date: February 1, 2018Inventors: Neal Wadhwa, Tali Dekel, Donglai Wei, Frederic Durand, William T. Freeman
-
Patent number: 9842404Abstract: An imaging method and corresponding apparatus according to an embodiment of the present invention enables measurement and visualization of fluid flow. An embodiment method includes obtaining video captured by a video camera with an imaging plane. Representations of motions in the video are correlated. A textured background of the scene can be modeled as stationary, with a refractive field translating between background and video camera. This approach offers multiple advantages over conventional fluid flow visualization, including an ability to use ordinary video equipment outside a laboratory without particle injection. Even natural backgrounds can be used, and fluid motion can be distinguished from refraction changes. Depth and three-dimensional information can be recovered using stereo video, and uncertainty methods can enhance measurement robustness where backgrounds are less textured. Example applications can include avionics and hydrocarbon leak detection.Type: GrantFiled: May 15, 2014Date of Patent: December 12, 2017Assignee: Massachusetts Institite of TechnologyInventors: William T. Freeman, Frederic Durand, Tianfan Xue, Michael Rubinstein, Neal Wadhwa
-
Patent number: 9805475Abstract: In one embodiment, a method of amplifying temporal variation in at least two images includes converting two or more images to a transform representation. The method further includes, for each spatial position within the two or more images, examining a plurality of coefficient values. The method additionally includes calculating a first vector based on the plurality of coefficient values. The first vector can represent change from a first image to a second image of the at least two images describing deformation. The method also includes modifying the first vector to create a second vector. The method further includes calculating a second plurality of coefficients based on the second vector.Type: GrantFiled: September 7, 2012Date of Patent: October 31, 2017Assignees: Massachusetts Institute of Technology, Quanta Computer Inc.Inventors: Michael Rubinstein, Neal Wadhwa, Frederic Durand, William T. Freeman, Hao-yu Wu, Eugene Inghaw Shih, John V. Guttag
-
Publication number: 20170221216Abstract: A method and corresponding device for identifying operational mode shapes of an object in a video stream includes extracting pixel-wise Eulerian motion signals of an object from an undercomplete representation of frames within a video stream. Pixel-wise Eulerian motion signals are downselected to produce a representative set of Eulerian motion signals of the object. Operational mode shapes of the object are identified based on the representative set. Resonant frequencies can also be identified. Embodiments enable vibrational characteristics of objects to be determined using video in near real time.Type: ApplicationFiled: February 1, 2016Publication date: August 3, 2017Inventors: Justin Gejune Chen, Oral Buyukozturk, William T. Freeman, Frederic Pierre Durand, Myers Abraham Davis, Neal Wadhwa
-
Publication number: 20170220718Abstract: Structural health monitoring (SHM) is essential but can be expensive to perform. In an embodiment, a method includes sensing vibrations at a plurality of locations of a structure by a plurality of time-synchronized sensors. The method further includes determining a first set of dependencies of all sensors of the time-synchronized sensors at a first sample time to any sensors of a second sample time, and determining a second set of dependencies of all sensors of the time-synchronized sensors at the second sample time to any sensors of a third sample time. The second sample time is later than the first sample time, and the third sample time is later than the second sample time. The method then determines whether the structure has changed if the first set of dependencies is different from the second set of dependencies. Therefore, automated SHM can ensure safety at a lower cost to building owners.Type: ApplicationFiled: February 1, 2016Publication date: August 3, 2017Inventors: William T. Freeman, Oral Buyukozturk, John W. Fisher, III, Frederic Durand, Hossein Mobahi, Neal Wadhwa, Zoran Dzunic, Justin G. Chen, James Long, Reza Mohammadi Ghazi, Theodericks Johannes Smit, Sergio Daniel Kapusta
-
Patent number: 9710917Abstract: An imaging method and corresponding apparatus according to an embodiment of the present invention enables measurement and visualization of fluid flow. An embodiment method includes obtaining video captured by a video camera with an imaging plane. Representations of motions in the video are correlated. A textured background of the scene can be modeled as stationary, with a refractive field translating between background and video camera. This approach offers multiple advantages over conventional fluid flow visualization, including an ability to use ordinary video equipment outside a laboratory without particle injection. Even natural backgrounds can be used, and fluid motion can be distinguished from refraction changes. Depth and three-dimensional information can be recovered using stereo video, and uncertainty methods can enhance measurement robustness where backgrounds are less textured. Example applications can include avionics and hydrocarbon leak detection.Type: GrantFiled: May 15, 2014Date of Patent: July 18, 2017Assignee: Massachusetts Institute of TechnologyInventors: William T. Freeman, Frederic Durand, Tianfan Xue, Michael Rubinstein, Neal Wadhwa
-
Patent number: 9338331Abstract: Some embodiments are directed to a method, corresponding system, and corresponding apparatus for rendering a video and/or image display to amplify small motions through video magnification. Some embodiments include a new compact image pyramid representation, the Riesz pyramid, that may be used for real-time, high-quality phase-based video magnification. Some embodiments are less overcomplete than even the smallest two orientation, octave-bandwidth complex steerable pyramid. Some embodiments are implemented using compact, efficient linear filters in the spatial domain. Some embodiments produce motion magnified videos that are of comparable quality to those using the complex steerable pyramid. In some embodiments, the Riesz pyramid is used with phase-based video magnification. The Riesz pyramid may phase-shift image features along their dominant orientation, rather than along every orientation like the complex steerable pyramid.Type: GrantFiled: January 8, 2015Date of Patent: May 10, 2016Assignee: Massachusetts Institute of TechnologyInventors: Neal Wadhwa, Michael Rubinstein, Frederic Durand, William T. Freeman
-
Patent number: 9324005Abstract: In one embodiment, a method of amplifying temporal variation in at least two images includes converting two or more images to a transform representation. The method further includes, for each spatial position within the two or more images, examining a plurality of coefficient values. The method additionally includes calculating a first vector based on the plurality of coefficient values. The first vector can represent change from a first image to a second image of the at least two images describing deformation. The method also includes modifying the first vector to create a second vector. The method further includes calculating a second plurality of coefficients based on the second vector.Type: GrantFiled: December 6, 2012Date of Patent: April 26, 2016Assignee: Massachusetts Institute of Technology Quanta Computer Inc.Inventors: Neal Wadhwa, Michael Rubinstein, Frederic Durand, William T. Freeman, Hao-Yu Wu, Eugene Inghaw Shih, John V. Guttag