RECORDING ANIMATION OF RIGID OBJECTS USING A SINGLE 3D SCANNER
This application teaches a method or methods related to recording animation. Such a method may include determining a reference model of an object by separating a 3D image of the object from a 3D image of its environment. The method may also include analyzing the reference model using a feature detection and localization algorithm(s). The object may then be recorded in motion, and the recording may be analyzed using feature detection and localization algorithms(s). Features of the recording may be matched to features of the reference model, wherein a match between the reference model and a frame of the recording comprises a pose of the object. A video animation may be created by recording a time series of poses of the object.
Latest KNOCKOUT CONCEPTS, LLC Patents:
- Graphical feedback during 3D scanning operations for obtaining optimal scan resolution
- IDENTIFYING OBJECTS USING A 3D SCANNING DEVICE, IMAGES, AND 3D MODELS
- GRAPHICAL FEEDBACK DURING 3D SCANNING OPERATIONS FOR OBTAINING OPTIMAL SCAN RESOLUTION
- Active Point Cloud Modeling
- 3D DATA TO 2D AND ISOMETRIC VIEWS FOR LAYOUT AND CREATION OF DOCUMENTS
A. Field of Invention
Some embodiments may generally relate to the field of extracting elements of 3D images in motion.
B. Description of the Related Art
Various video recording methodologies are known in the art as well as various methods of computer analysis of video. However, current recording analysis technologies tend to confine users to merely recognizing features in image data. Furthermore, objects in recorded digital video cannot be manipulated as in the manner of a 3D CAD drawing. What is missing is methodology for separating an object from its background in a 3D reconstructed model of a static scene, then using video of the same object in motion to obtain further structural detail of the object, and creating a 3D model object that can be reoriented, manipulated, and moved independent of the image or video from which it was created.
Some embodiments of the present invention may provide one or more benefits or advantages over the prior art.
II. SUMMARY OF THE INVENTIONSome embodiments may relate to a method for recording animation comprising the steps of: determining a reference model of an object by separating a three-dimensional model of the object from its environment in a 3D reconstruction of a static scene; analyzing the reference model using a feature detection and localization algorithm; recording movement of the object; analyzing the recording using feature detection and localization algorithms; matching features of the recording to features of the reference model, wherein a match between the reference model and a frame of the recording comprises a pose of the object; and recording a time series of poses of the object, the time series comprising an animation.
Embodiments may further comprise the step of saving the reference model in association with the animation on a computer readable medium.
According to some embodiments data for determining the reference model is obtained with a three-dimensional scanning device.
According to some embodiments the step of separating the three-dimensional model of the object from its environment is conducted by the three-dimensional scanning device.
According to some embodiments the step of analyzing the reference model is conducted by the three dimensional scanning device.
According to some embodiments the data for determining the reference model of the object, and from recording movement of the object, are obtained with the same three-dimensional scanning device.
According to some embodiments the feature detection and localization algorithm for analyzing the reference model is selected from one or more of RANSAC, iterative closest point, a least squares method, a Newtonian method, a quasi-Newtonian method, or an expectation-maximization method, detection of principal curvatures, or detection of distance to a medial surface.
According to some embodiments the feature detection and localization algorithm for analyzing the recording is selected from one or more of RANSAC, iterative closest point, a least squares method, a Newtonian method, a quasi-Newtonian method, or an expectation-maximization method, detection of principal curvatures, or detection of distance to a medial surface.
According to some embodiments a quantity of digital computations of a microprocessor is reduced by applying a Kalman filter to the step of analyzing the recording using feature detection and localization algorithms.
Embodiments may also relate to a method for recording animation comprising the steps of: determining a reference model of an object by separating a three-dimensional reconstruction of the object from its environment in a 3D reconstruction of a static scene; analyzing the reference model using a feature detection and localization algorithm selected from one or more of RANSAC, iterative closest point, a least squares method, a Newtonian method, a quasi-Newtonian method, or an expectation-maximization method, detection of principal curvatures, or detection of distance to a medial surface; recording movement of the object; analyzing the recording using feature detection and localization algorithms selected from one or more of RANSAC, iterative closest point, a least squares method, a Newtonian method, a quasi-Newtonian method, or an expectation-maximization method, detection of principal curvatures, or detection of distance to a medial surface, wherein a quantity of digital computations of a microprocessor is reduced by applying a Kalman filter; matching features of the recording to features of the reference model, wherein a match between the reference model and a frame of the recording comprises a pose of the object; and recording a time series of poses of the object, the time series comprising an animation.
Embodiments may also relate to a method for recording animation comprising the steps of: determining a reference model of an object by separating a three-dimensional reconstruction of the object from its environment in a 3D reconstruction of a static scene; analyzing the reference model using a feature detection and localization algorithm selected from one or more of RANSAC, iterative closest point, a least squares method, a Newtonian method, a quasi-Newtonian method, or an expectation-maximization method, detection of principal curvatures, or detection of distance to a medial surface; recording movement of the object; analyzing the recording using feature detection and localization algorithms selected from one or more of RANSAC, iterative closest point, a least squares method, a Newtonian method, a quasi-Newtonian method, or an expectation-maximization method, detection of principal curvatures, or detection of distance to a medial surface, wherein a quantity of digital computations of a microprocessor is reduced by applying a Kalman filter; matching features of the recording to features of the reference model, wherein a match between the reference model and a frame of the recording comprises a pose of the object; and recording a time series of poses of the object, the time series comprising an animation; wherein the step of separating the three-dimensional image of the object from the three-dimensional image of the environment of the object is conducted by the three-dimensional scanning device, and wherein the data for determining the reference model of the object, and from recording movement of the object, are obtained with the same three-dimensional scanning device.
Other benefits and advantages will become apparent to those skilled in the art to which it pertains upon reading and understanding of the following detailed specification.
The invention may take physical form in certain parts and arrangement of parts, embodiments of which will be described in detail in this specification and illustrated in the accompanying drawings which form a part hereof and wherein:
A method for recording animation of a three-dimensional real world object includes separating a 3D model of the object from a 3D model of its surroundings. Many known 3D scanners and cameras are capable of achieving obtaining the data necessary for method according to embodiments of this invention. This model of the 3D object separated from the model of its environment, may be used as a reference model. The reference model may be further analyzed using a feature detection and localization algorithm to identify various features of the reference model that may be used for comparison with live feed from the 3D scanning device. Movement, manually induced or otherwise, of the object may be recorded using the 3D scanning device. Once again the features of the recording of the object in motion may be analyzed utilizing similar feature detection and localization algorithms. The features of the recording can be compared with the features of the reference model, and when matches are found said matches may comprise poses of the object for rendering an animation. Finally, the poses may be recombined in any order to formulate an animation of the object. The combination of a time series of poses arranged in any order and an arbitrary background allows one to create animations of the object that differ from the motion observed in the previously recorded video. As used herein the term posed includes the generally accepted meaning in the 3D imaging arts.
Referring now to the drawings wherein the showings are for purposes of illustrating embodiments of the invention only and not for purposes of limiting the same,
The reference model may analyzed using feature detection and localization algorithms 112 in order to enable later comparison of the features and related data with live feed from the scanning device. The feature detection and localization algorithm used for analyzing the reference model may be chosen from many processes and algorithms now known or developed in the future. Some such feature detection and localization algorithms include RANSAC (Random Sample Consensus), iterative closest point, least squares methods, Newtonian methods, quasi-Newtonian methods, expectation-maximization methods, detection of principal curvatures, or detection of distance to a medial surface. The methodology and corresponding algorithms of all of these processes are known in the art and incorporated by reference herein. In an illustrative embodiment, during the step of analyzing the recording using a feature detection and localization algorithm, the quantity of digital computations of a microprocessor may be reduced by applying a Kalman filter. In this context a Kalman filter allows embodiments to accurately predict the next position and/or orientation of the object which enables embodiments to apply feature detection calculations to smaller regions of the 3D data. Kalman filter methodology is known in the art and is incorporated by reference herein.
Movement of the real world three-dimensional object may be manually induced and recorded using a 3D scanning device 114. Features of the object in the recording may be analyzed using similar feature detection and localization algorithms 116. The feature detection and localization algorithm used for analyzing the recording may be chosen from many processes and algorithms now known or developed in the future. Some such feature detection and localization algorithms include RANSAC (Random Sample Consensus), iterative closest point, least squares methods, Newtonian methods, quasi-Newtonian methods, expectation-maximization methods, detection of principal curvatures, or detection of distance to a medial surface. The methodology and corresponding algorithms of all of these processes are incorporated by reference herein. In an illustrative embodiment, during the step of analyzing the recording using a feature detection and localization algorithm, the quantity of digital computations of a microprocessor may be reduced by applying a Kalman filter.
Once the features of the recording are obtained, such features may be compared with the features of the reference model 118. A match between the features of the recording and the features of the reference model comprises a pose of the object. The feature comparison may be continuously made until multiple matches result in multiple poses 120 being obtained. In an alternate embodiment, the matching of the features to obtain poses is done in real time when the recording is being made. A time series of the various poses may be recorded in any order comprising an animation of the object 122. In an illustrative embodiment, the reference model initially obtained may be saved in association with the animation. This may be saved on any computer readable medium.
It will be apparent to those skilled in the art that the above methods and apparatuses may be changed or modified without departing from the general scope of the invention. The invention is intended to include all such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Having thus described the invention, it is now claimed:
Claims
1. A method for recording animation comprising the steps of:
- determining a reference model of an object by separating a three-dimensional model of the object from its environment in a 3D reconstruction of a static scene;
- analyzing the reference model using a feature detection and localization algorithm;
- recording movement of the object;
- analyzing the recording using feature detection and localization algorithms;
- matching features of the recording to features of the reference model, wherein a match between the reference model and a frame of the recording comprises a pose of the object; and
- recording a time series of poses of the object, the time series comprising an animation.
2. The method of claim 1 further comprising the step of saving the reference model in association with the animation on a computer readable medium.
3. The method of claim 1, wherein data for determining the reference model is obtained with a three-dimensional scanning device.
4. The method of claim 3, wherein the step of separating the three-dimensional image of the object from the three-dimensional image of the environment of the object is conducted by the three-dimensional scanning device.
5. The method of claim 3, wherein the step of analyzing the reference model is conducted by the three dimensional scanning device.
6. The method of claim 3, wherein the data for determining the reference model of the object, and for recording movement of the object, are obtained with the same three-dimensional scanning device.
7. The method of claim 1, wherein the feature detection and localization algorithm for analyzing the reference model is selected from one or more of RANSAC, iterative closest point, a least squares method, a Newtonian method, a quasi-Newtonian method, or an expectation-maximization method, detection of principal curvatures, or detection of distance to a medial surface.
8. The method of claim 1, wherein the feature detection and localization algorithm for analyzing the recording is selected from one or more of RANSAC, iterative closest point, a least squares method, a Newtonian method, a quasi-Newtonian method, or an expectation-maximization method, detection of principal curvatures, or detection of distance to a medial surface.
9. The method of claim 1, wherein a quantity of digital computations of a microprocessor is reduced by applying a Kalman filter to the step of analyzing the recording using feature detection and localization algorithms.
10. A method for recording animation comprising the steps of:
- determining a reference model of an object by separating a three-dimensional model of the object from its an environment in a 3D reconstruction of a static scene;
- analyzing the reference model using a feature detection and localization algorithm selected from one or more of RANSAC, iterative closest point, a least squares method, a Newtonian method, a quasi-Newtonian method, or an expectation-maximization method, detection of principal curvatures, or detection of distance to a medial surface;
- recording movement of the object;
- analyzing the recording using feature detection and localization algorithms selected from one or more of RANSAC, iterative closest point, a least squares method, a Newtonian method, a quasi-Newtonian method, or an expectation-maximization method, detection of principal curvatures, or detection of distance to a medial surface, wherein a quantity of digital computations of a microprocessor is reduced by applying a Kalman filter;
- matching features of the recording to features of the reference model, wherein a match between the reference model and a frame of the recording comprises a pose of the object; and
- recording a time series of poses of the object, the time series comprising an animation.
11. A method for recording animation comprising the steps of:
- determining a reference model of an object by separating a three-dimensional model of the object from its environment in a 3D reconstruction of a static scene;
- analyzing the reference model using a feature detection and localization algorithm selected from one or more of RANSAC, iterative closest point, a least squares method, a Newtonian method, a quasi-Newtonian method, or an expectation-maximization method, detection of principal curvatures, or detection of distance to a medial surface;
- recording movement of the object;
- analyzing the recording using feature detection and localization algorithms selected from one or more of RANSAC, iterative closest point, a least squares method, a Newtonian method, a quasi-Newtonian method, or an expectation-maximization method, detection of principal curvatures, or detection of distance to a medial surface, wherein a quantity of digital computations of a microprocessor is reduced by applying a Kalman filter;
- matching features of the recording to features of the reference model, wherein a match between the reference model and a frame of the recording comprises a pose of the object; and
- recording a time series of poses of the object, the time series comprising an animation;
- wherein the step of separating the three-dimensional image of the object from the three-dimensional image of the environment of the object is conducted by the three-dimensional scanning device, and wherein the data for determining the reference model of the object, and from recording movement of the object, are obtained with the same three-dimensional scanning device.
12. The method of claim 11, wherein the step of separating the three-dimensional image of the object from the three-dimensional image of the environment of the object is conducted by the three-dimensional scanning device.
13. The method of claim 12, wherein the step of analyzing the reference model is conducted by the three dimensional scanning device.
Type: Application
Filed: Mar 27, 2015
Publication Date: Oct 1, 2015
Applicant: KNOCKOUT CONCEPTS, LLC (Columbus, OH)
Inventors: Stephen Brooks Myers (Shreve, OH), Jacob Abraham Kuttothara (Loudonville, OH), Steven Donald Paddock (Richfield, OH), John Moore Wathen (Akron, OH), Andrew Slatton (Columbus, OH)
Application Number: 14/671,313