SYSTEM FOR CHARACTERIZING MANUAL WELDING OPERATIONS ON PIPE AND OTHER CURVED STRUCTURES

Abstract

A system for characterizing manual welding exercises and providing valuable training to welders that includes components for generating, capturing, and processing data. The data generating component further includes a fixture, workpiece, at least one calibration device having at least two point markers integral therewith, and a welding tool. The data capturing component further includes an imaging system for capturing images of the point markers and the data processing component is operative to receive information from the data capturing component and perform various position and orientation calculations.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is being filed as a non-provisional patent application claiming priority under 35 U.S.C. §119(e) from, and any other benefit of, U.S. Provisional Patent Application No. 62/055,724 filed on Sep. 26, 2014, the entire disclosure of which is herein incorporated by reference.

The following commonly-assigned, co-pending U.S. patent application is also incorporated by reference herein in its entirety: U.S. Non-Provisional patent application Ser. No. 13/543,240, filed on Jul. 6, 2012 and entitled “System for Characterizing Manual Welding Operations.”

FIELD

The general inventive concepts relate, among other things, to methods, apparatus, systems, programs, and techniques for remotely controlling operation of a welding device.

BACKGROUND

The described invention relates in general to a system for characterizing manual welding operations, and more specifically to a system for providing useful information to a welding trainee by capturing, processing, and presenting in a viewable format, data generated by the welding trainee in manually executing an actual weld in real time.

The manufacturing industry's desire for efficient and economical welder training has been a well documented topic over the past decade as the realization of a severe shortage of skilled welders is becoming alarmingly evident in today's factories, shipyards, and construction sites. A rapidly retiring workforce, combined with the slow pace of traditional instructor-based welder training has been the impetus for the development of more effective training technologies Innovations which allow for the accelerated training of the manual dexterity skills specific to welding, along with the expeditious indoctrination of arc welding fundamentals are becoming a necessity. The characterization and training system disclosed herein addresses this vital need for improved welder training and enables the monitoring of manual welding processes to ensure the processes are within permissible limits necessary to meet industry-wide quality requirements. To date, the majority of welding processes are performed manually, yet the field is lacking practical commercially available tools to track the performance of these manual processes. Thus, there is an ongoing need for an effective system for training welders to properly execute various types of welds under various conditions.

SUMMARY

The following provides a summary of certain exemplary embodiments of the present invention. This summary is not an extensive overview and is not intended to identify key or critical aspects or elements of the present invention or to delineate its scope.

In accordance with one aspect of the present invention, a system for characterizing manual and/or semiautomatic welding operations and exercises is provided. This system includes a data generating component; a data capturing component; and a data processing component. The data generating component further includes a fixture, wherein the geometric characteristics of the fixture are predetermined; a workpiece adapted to be mounted on the fixture, wherein the workpiece includes at least one joint to be welded, and wherein the vector extending along the joint to be welded defines an operation path, wherein the operation path is linear, curvilinear, circular, or a combination thereof; at least one calibration device, wherein each calibration device further includes at least two point markers integral therewith, and wherein the geometric relationship between the point markers and the operation path is predetermined; and a welding tool, wherein the welding tool is operative to form a weld at the joint to be welded, wherein the welding tool defines a tool point and a tool vector, and wherein the welding tool further includes a target attached to the welding tool, wherein the target further includes a plurality of point markers mounted thereon in a predetermined pattern, and wherein the predetermined pattern of point markers is operative to define a rigid body. The data capturing component further includes an imaging system for capturing images of the point markers. The data processing component is operative to receive information from the data capturing component and then calculate the position and orientation of the operation path relative to the three-dimensional space viewable by the imaging system; the position of the tool point and orientation of the tool vector relative to the rigid body; and the position of the tool point and orientation of the tool vector relative to the operation path.

In accordance with another aspect of the present invention, a system for characterizing manual and/or semiautomatic welding operations and exercises is also provided. This system includes a data generating component; a data capturing component; and a data processing component. The data generating component further includes a fixture, wherein the geometric characteristics of the fixture are predetermined; a workpiece adapted to be mounted on the fixture, wherein the workpiece includes at least one joint to be welded, and wherein the vector extending along the joint to be welded defines an operation path, wherein the operation path is linear, curvilinear, circular, or a combination thereof; at least one calibration device, wherein each calibration device further includes at least two point markers integral therewith, and wherein the geometric relationship between the point markers and the operation path is predetermined; and a welding tool, wherein the welding tool is operative to form a weld at the joint to be welded, wherein the welding tool defines a tool point and a tool vector, and wherein the welding tool further includes a target attached to the welding tool, wherein the target further includes a plurality of point markers mounted thereon in a predetermined pattern, and wherein the predetermined pattern of point markers is operative to define a rigid body. The data capturing component further includes an imaging system for capturing images of the point markers and the imaging system further includes a plurality of digital cameras. At least one band-pass filter is incorporated into the optical sequence for each of the plurality of digital cameras for permitting light from only the wavelengths which are reflected or emitted from the point markers for improving image signal-to-noise ratio. The data processing component is operative to receive information from the data capturing component and then calculate the position and orientation of the operation path relative to the three-dimensional space viewable by the imaging system; the position of the tool point and orientation of the tool vector relative to the rigid body; and the position of the tool point and orientation of the tool vector relative to the operation path.

Additional features and aspects of the present invention will become apparent to those of ordinary skill in the art upon reading and understanding the following detailed description of the exemplary embodiments. As will be appreciated by the skilled artisan, further embodiments of the invention are possible without departing from the scope and spirit of the invention. Accordingly, the drawings and associated descriptions are to be regarded as illustrative and not restrictive in nature.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated into and form a part of the specification, schematically illustrate one or more exemplary embodiments of the invention and, together with the general description given above and detailed description given below, serve to explain the principles of the invention, and wherein:

FIG. 1 is a flow chart illustrating the flow of information through the data processing and visualization component of an exemplary embodiment of the present invention.

FIG. 2 provides an isometric view of a portable or semi-portable system for characterizing manual welding operations, in accordance with an exemplary embodiment of the present invention.

FIG. 3 provides an isometric view of the flat assembly of the system of FIG. 2.

FIG. 4 provides an isometric view of the horizontal assembly of the system of FIG. 2.

FIG. 5 provides an isometric view of the vertical assembly of the system of FIG. 2.

FIG. 6 illustrates the placement of two point markers on the flat assembly of FIG. 2.

FIG. 7 illustrates an exemplary workpiece operation path.

FIG. 8 illustrates the placement of two active or passive point markers on an exemplary workpiece for determining a workpiece operation path.

FIG. 9 is a flowchart detailing the process steps involved in an exemplary embodiment of a first calibration component of the present invention.

FIG. 10 illustrates the welding tool of an exemplary embodiment of this invention showing the placement of the point markers used to define the rigid body.

FIG. 11 illustrates the welding tool of an exemplary embodiment of this invention showing the placement of the point markers used to define the tool vector and the rigid body.

FIG. 11A illustrates the welding tool of FIG. 10, along with an exemplary tool calibration device for interfacing thererwith.

FIG. 12 is a flowchart detailing the process steps involved in an exemplary embodiment of a second calibration component of the present invention.

FIG. 13 provides an isometric view of a portable or semi-portable system for characterizing manual welding operations, in accordance with an exemplary embodiment of the present invention.

DETAILED DESCRIPTION

Exemplary embodiments of the present invention are now described with reference to the Figures. Reference numerals are used throughout the detailed description to refer to the various elements and structures. In other instances, well-known structures and devices are shown in block diagram form for purposes of simplifying the description. Although the following detailed description contains many specifics for the purposes of illustration, a person of ordinary skill in the art will appreciate that many variations and alterations to the following details are within the scope of the invention. Accordingly, the following embodiments of the invention are set forth without any loss of generality to, and without imposing limitations upon, the claimed invention.

The present invention relates to an advanced system for observing and characterizing manual welding exercises and operations. This system is particularly useful for welding instruction and welder training that provides an affordable tool for measuring manual welding technique and comparing that technique with established procedures. The training applications of this invention include: (i) screening applicant skill levels; (ii) assessing trainee progress over time; (iii) providing real-time coaching to reduce training time and costs; and (iv) periodically re-testing welder skill levels with quantifiable results. Processing monitoring and quality control applications include: (i) identification of deviations from preferred conditions in real time; (ii) documenting and tracking compliance with procedures over time; (iii) capturing in-process data for statistical process control purposes (e.g., heat input measurements); and (iv) identifying welders needing additional training The system of the present invention provides the unique benefit of enabling the determination of compliance with various accepted welding procedures.

The present invention, in various exemplary embodiments, measures torch motion and gathers process data during welding exercises using a single or multiple camera tracking system based on point cloud image analysis. This invention is applicable to a wide range of processes including, but not necessarily limited to, GMAW, FCAW, SMAW, GTAW, and cutting. The invention is expandable to a range of workpiece configurations, including large sizes, various joint types, pipe, plate, and complex shapes and assemblies. Measured parameters may include, but are not limited to, work angle, travel angle, tool standoff, travel speed, bead placement, weave, voltage, current, wire feed speed, arc length, heat input, gas flow (metered), contact tip to work distance (CTWD), and deposition rate (e.g., lbs./hr., in./run). The training component of the present invention may be pre-populated with specific welding procedures or it may be customized by an instructor. Data is automatically saved and recorded, a post-weld analysis scores performance, and progress is tracked over time. This system may be used throughout an entire welding training program. The system may be used to provide any type of feedback (typically in real time) including, but not limited to, one or more of in-helmet visual feedback, on-screen visual feedback, audio feedback (e.g., coaching), and welding tool (e.g., torch) visual, audio, or tactile feedback. With reference now to the Figures, one or more specific embodiments of this invention shall be described in greater detail.

As shown in FIG. 1, in an exemplary embodiment of the present invention, the basic flow of information through data generating component 100, data capturing component 200, and data processing (and visualization) component 300 of weld characterization system 10 occurs in six basic steps: (1) image capture 110; (2) image processing 112; (3) input of arc weld data 210, such as known or preferred weld parameters; (4) data processing 212; (5) data storage 214; and (5) data display 310. Image capture step 110 includes capturing images of a target 98 (which typically includes at least two point markers located in a fixed geometric relationship to one another) with one or more off-the-shelf high-speed-vision cameras, where the output aspect typically includes creating an image file at many (e.g., over 100) frames per second. The input aspect of image processing step 112 includes frame-by-frame point cloud analysis of a rigid body that includes three or more point markers (i.e., the calibrated target). Upon recognition of a known rigid body, position and orientation are calculated relative to the camera origin and the “trained” rigid body orientation. Capturing and comparing the images from two or more cameras allows for a substantially accurate determination of the rigid body position and orientation in three-dimensional space. Images are typically processed at a rate of more than 100 times per second. One of ordinary skill in the art will appreciate that a lesser sampling rate (e.g., 10 images/sec.) or a greater sampling rate (e.g., 1,000 images/sec.) could be used. The output aspect of image processing step 112 includes creation of a data array that includes x-axis, y-axis, and z-axis positional data and roll, pitch, and yaw orientation data, as well as time stamps and software flags. The text file (including the aforementioned 6D data) may be streamed or sent at a desired frequency. The input aspect of data processing step 212 includes raw positional and orientation data typically requested at a predetermined rate, while the output aspect includes transforming this raw data into useful welding parameters with algorithms specific to a selected process and joint type. The input aspect of data storage step 214 includes storing welding trial data (e.g., as a *.dat file), while the output aspect includes saving the data for review and tracking, saving the data for review on a monitor at a later time, and/or reviewing the progress of the student at a later time. Student progress may include, but is not limited to, total practice time, total arc time, total arc starts, and individual parameter-specific performance over time. The input aspect of data display step 310 includes welding trial data that may further include, but is not limited to, work angle, travel angle, tool standoff, travel speed, bead placement, weave, voltage, current, wire-feed speed, arc length, heat input, gas flow (metered), contact tip to work distance (CTWD), and deposition rate. The output aspect involves any type of feedback including, but not limited to, one or more of visual data that may be viewed on a monitor, in-helmet display, heads-up display, or combinations thereof; audio data (e.g., audio coaching); and tactile feedback. The feedback is typically presented in real time. The tracked parameters are plotted on a time-based axis and compared to upper and lower thresholds or preferred variations, such as those trained by recording the motions of an expert welder. One of ordinary skill in the art will appreciate that the general inventive concepts contemplate plotting the parameters on an axis that is not time-based, such as plotting or otherwise processing the parameters based on a length of the weld. Current and voltage may be measured in conjunction with travel speed to determine heat input, and the welding process parameters may be used to estimate arc length. Position data may be transformed into weld start position, weld stop position, weld length, weld sequence, welding progression, or combinations thereof and current and voltage may be measured in conjunction with travel speed to determine heat input.

FIGS. 2-5 provide illustrative views of weld characterization system 10 in accordance with an exemplary embodiment the present invention. As shown in FIG. 2, portable training stand 20 includes a substantially flat base 22 for contact with a floor or other horizontal substrate, rigid vertical support column 24, camera or imaging device support 26, and rack and pinion assembly 31 for adjusting the height of imaging device support 26. In most embodiments, weld characterization system 10 is intended to be portable or at least moveable from one location to another, therefore the overall footprint of base 22 is relatively small to permit maximum flexibility with regard to installation and use. As shown in FIG. 2-6, weld characterization system 10 may be used for training exercises that include any suitable arrangement of workpieces including, but not limited to, flat, horizontally, vertically, overhead, and out-of-position oriented workpieces. In the exemplary embodiments shown in the Figures, training stand 20 is depicted as a unitary or integrated structure that is capable of supporting the other components of system. In other embodiments, stand 20 is absent and the various components of the system are supported by whatever suitable structural or supportive means may be available. Thus, within the context of this invention, “stand” 20 is defined as any single structure or, alternately, multiple structures that are capable of supporting the components of weld characterization system 10.

With reference to FIGS. 2-3, certain welding exercises will utilize a flat assembly 30, which is slidably attached to vertical support column 24 by collar 34, which slides upward or downward on support column 24. Collar 34 is further supported on column 24 by rack and pinion 31, which includes shaft 32 for moving rack and pinion assembly 31 upward or downward on support column 24. Flat assembly 30 includes training platform 38, which is supported by one or more brackets (not visible). In some embodiments, a shield 42 is attached to training platform 38 for protecting the surface of support column 24 from heat damage. Training platform 38 further includes at least one clamp 44 for securing weld position-specific fixture/jig 46 to the surface of the training platform. The structural configuration or general characteristics of weld position-specific jig 46 are variable based on the type of weld process that is the subject of a particular welding exercise, and in FIGS. 2-3, fixture 46 is configured for a fillet weld exercise. In the exemplary embodiment shown in FIGS. 2-3, first 48 and second 50 structural components of weld position-specific fixture 46 are set at right angles to one another. Position-specific fixture 46 may include one or more pegs 47 for facilitating proper placement of a weld coupon (workpiece) 54 on the fixture. The characteristics of any weld coupon 54 used with system 10 are variable based on the type of manual welding process that is the subject of a particular training exercise and in the exemplary embodiment shown in the FIGS. 7-8, first 56 and second 58 portions of weld coupon 54 are also set at right angles to one another. With reference to FIGS. 4-5, certain other welding exercises will utilize a horizontal assembly 30 (see FIG. 4) or a vertical assembly 30 (see FIG. 5). In FIG. 4, horizontal assembly 30 supports butt fixture 46, which holds workpiece 54 in the proper position for a butt weld exercise. In FIG. 5, vertical assembly 30 supports vertical fixture 46, which holds workpiece 54 in the proper position for a lap weld exercise.

Data processing component 300 of the present invention typically includes at least one computer for receiving and analyzing information captured by the data capturing component 200, which itself includes at least one digital camera contained in a protective housing. During operation of weld characterization system 10, this computer is typically running software that includes a training regimen module, an image processing and rigid body analysis module, and a data processing module. The training regimen module includes a variety of weld types and a series of acceptable welding process parameters associated with creating each weld type. Any number of known or AWS weld joint types and the acceptable parameters associated with these weld joint types may be included in the training regimen module, which is accessed and configured by a course instructor prior to the beginning of a training exercise. The weld process and/or type selected by the instructor determine which weld process-specific fixture, calibration device, and weld coupon are used for any given training exercise. The object recognition module is operative to train the system to recognize a known rigid body target 98 (which includes two or more point markers) and then to use target 98 to calculate positional and orientation data for welding gun 90 as an actual manual weld is completed by a trainee. The data processing module compares the information in the training regimen module to the information processed by the object recognition module and outputs the comparative data to a display device such as a monitor or heads-up display. The monitor allows the trainee to visualize the processed data in real time and the visualized data is operative to provide the trainee with useful feedback regarding the characteristics and quality of the weld. The visual interface of weld characterization system 10 may include a variety of features related to the input of information, login, setup, calibration, practice, analysis, and progress tracking The analysis screen typically displays the welding parameters found in the training regimen module, including (but not limited to) work angle, travel angle, tool standoff, travel speed, bead placement, weave, voltage, current, wire-feed speed, arc length, heat input, gas flow (metered), contact tip to work distance (CTWD), and deposition rate (e.g., lbs./hr., in./run). Multiple display variations are possible with the present invention.

In most, if not all instances, weld characterization system 10 will be subjected to a series of calibration steps/processes prior to use. Some of the aspects of the system calibration will typically be performed by the manufacturer of system 10 prior to delivery to a customer and other aspects of the system calibration will typically be performed by the user of weld characterization system 10 prior to any welding training exercises. System calibration typically involves two related and integral calibration processes: (i) determining the three-dimensional position and orientation of the operation path to be created on a workpiece for each joint/position combination to be used in various welding training exercises; and (ii) determining the three-dimensional position and orientation of the welding tool by calculating the relationship between a plurality of passive (e.g., reflective) or active (e.g., light emitting) point markers located on target 98 and at least two key points represented by point markers located on the welding tool 90.

The first calibration aspect of this invention typically involves the calibration of the welding operation with respect to the global coordinate system, i.e., relative to the other structural components of weld characterization system 10 and the three-dimensional space occupied thereby. Prior to tracking/characterizing a manual welding exercise, the global coordinates of each desired operation path (i.e., vector) on any given workpiece will be determined. In some embodiments, this is a factory-executed calibration process that will include corresponding configuration files stored on data processing component 200. In other embodiments, the calibration process could be executed in the field (i.e., on site). To obtain the desired vectors, a calibration device containing active or passive markers may be inserted on at least two locating markers in each of the various platform positions (e.g., flat, horizontal, and vertical). FIGS. 6-8 illustrate this calibration step in one possible platform position. Joint-specific fixture 46 includes first and second structural components 48 (horizontal) and 50 (vertical), respectively. Weld coupon or workpiece 54 includes first and second portions 56 (horizontal) and 58 (vertical), respectively. One of ordinary skill in the art will appreciate that the general inventive concepts extend to any suitable jig/coupon configurations and orientations. For example, in some exemplary embodiments, jig or joint calibration could be performed using a handheld or removable device that would “teach” the software points (i.e., positions) to determine the operation path. In this manner, use of the two or more points would allow the weldment to be oriented in any position.

Workpiece operation path extends from point X to point Y and is shown as double line 59 in FIG. 7. Locating point markers 530 and 532 are placed as shown in FIGS. 6 (and FIG. 8), and the location of each marker is obtained using data capturing component 100. Any suitable data capturing system can be used, for example, the Optitrack Tracking Tools (provided by NaturalPoint, Inc. of Corvallis, Oregon) or a similar commercially available or proprietary hardware/software system that provides three-dimensional marker and six degrees of freedom object motion tracking in real time. Such technologies typically utilize reflective and/or light emitting point markers arranged in predetermined patterns to create point clouds that are interpreted by system imaging hardware and system software as “rigid bodies,” although other suitable methodologies are compatible with this invention.

In the calibration process represented by the flowchart of FIG. 9, table 38 is fixed into position i (0,1,2) at step 280; a calibration device is placed on locating pins at step 282; all marker positions are captured at step 284; coordinates for the locator positions are calculated at step 286; coordinates for the fillet operation path are calculated at step 288 and stored at 290; coordinates for the lap operation path are calculated at step 292 and stored at 294; and coordinates for the groove operation path are calculated at step 296 and stored at 298. All coordinates are calculated relative to the three dimensional space viewable by data capturing component 200.

In one embodiment of this invention, the position and orientation of the workpiece is calibrated through the application of two or more passive or active point markers to a calibration device that is placed at a known translational and rotational offset to a fixture that holds the workpiece at a known translational and rotational offset. In another embodiment of this invention, the position and orientation of the workpiece is calibrated through the application of two or more passive or active point markers to a fixture that holds the workpiece at a known translational and rotational offset. In still other embodiments, the workpiece is non-linear, and the position and orientation thereof may be mapped using a calibration tool with two or more passive or active point markers and stored for later use. The position and orientation of the workpiece operation path may undergo a pre-determined translational and rotational offset from its original calibration plane based on the sequence steps in the overall work operation.

In some exemplary embodiments, data on weldments in electronic format (e.g., rendered in CAD) are extracted and used in determining position and/or orientation of the workpiece. Additionally, an associated welding procedure specification (WPS) that specifies welding parameters for the weldment is also processed. In this manner, the system can map the CAD drawing and WPS for use in assessing (in real time) compliance with the WPS.

Important tool manipulation parameters such as position, orientation, velocity, acceleration, and spatial relationship to the workpiece operation path may be determined from the analysis of consecutive tool positions and orientations over time and the various workpiece operation paths described above. Tool manipulation parameters may be compared with pre-determined preferred values to determine deviations from known and preferred procedures. Tool manipulation parameters may also be combined with other manufacturing process parameters to determine deviations from preferred procedures, and these deviations may be used for assessing skill level, providing feedback for training, assessing progress toward a skill goal, or for quality control purposes. Recorded motion parameters relative to the workpiece operation path may be aggregated from multiple operations for statistical process control purposes. Deviations from preferred procedures may be aggregated from multiple operations for statistical process control purposes. Important tool manipulation parameters and tool positions and orientations with respect to the workpiece operation path may also be recorded for establishing a signature of an experienced operator's motions to be used as a baseline for assessing compliance with preferred procedures.

The second calibration aspect typically involves the calibration of the welding tool 90 with respect to the target 98. The welding tool 90 is typically a welding torch or gun or SMAW electrode holder, but may also be any number of other implements including a soldering iron, cutting torch, forming tool, material removal tool, paint gun, or wrench. With reference to FIGS. 10-11, welding gun/tool 90 includes tool point 91, nozzle 92, body 94, trigger 96, and target 98. A tool calibration device 93, which includes two integrated active or passive point markers in the A and B positions (see FIG. 11) is attached to, inserted into, or otherwise interfaced with the nozzle 92. For example, the tool point 91 can be machined off so that the tool calibration device 93 can be threaded into the nozzle 92 of the welding tool 90 for calibration purposes.

In another exemplary embodiment, the tool calibration device 93 is affixed to a sleeve 1100 as shown in FIG. 11A. The sleeve 1100 can be shaped and sized so as to fit over at least a portion of the nozzle 92 of the welding tool 90. In some embodiments, the sleeve 1100 can fit over the nozzle 92 without requiring removal of the tool point 91. In some exemplary embodiments, the sleeve 1100 fits over or otherwise interfaces with a part of the welding tool 90 other than the nozzle 92. The sleeve 1100 removably connects to the welding tool 90 in any suitable matter, for example, by friction fit, threads, etc. The sleeve 1100 can be shaped and sized so as to fit over a plurality of different welding tools, without requiring modification of said welding tools. In this manner, the sleeve 1100 and the attached tool calibration device 93 form a type of “universal” tool calibration device.

Additionally, a rigid body point cloud (i.e., a “rigid body”) is constructed by attaching active or passive point markers 502, 504, and 506 (and possibly additional point markers) to the upper surface of target 98.

As described herein, other point marker placements are possible and fall within the scope of the general inventive concepts. Target 98 may include a power input if the point markers used are active and require a power source. Data capturing component 200 uses a tracking system (e.g., the aforementioned Optitrack Tracking Tools) or similar hardware/software to locate the rigid body and point markers 522 (A) and 520 (B), which represent the location of a tool vector. These positions can be extracted from the software of system 10 and the relationship between point markers A and B and the rigid body can be calculated.

In the exemplary calibration process represented by the flowchart of FIG. 12, weld nozzle 92 and the contact tube 91 are removed at step 250; the calibration device is inserted into body 94 at step 252; weld tool 90 is placed in the working envelope and rigid body (designated as “S” in FIG. 12) and point markers A and B are captured by data capturing component 100; and the relationships between A and S and B and S are calculated at step 256; with relationship data for A_s being stored at 258 and relationship data for B_s being stored at 260.

In one embodiment of this invention, calibration of the tool point and tool vector is performed through the application of two or more passive or active point markers to the calibration device at locations along the tool vector with a known offset to the tool point. In another embodiment, calibration of the tool point and tool vector is performed by inserting the tool into a calibration block of known position and orientation relative to the workpiece. For example, calibration of the tool point and tool vector can be performed by inserting the tool point 91 into the weld joint in a particular manner.

With regard to the rigid body defined by the point markers (e.g., 502, 504, 506), in one embodiment, the passive or active point markers are affixed to the tool in a multi-faceted manner so that a wide range of rotation and orientation changes can be accommodated within the field of view of the imaging system. In another embodiment, the passive or active point markers are affixed to the tool in a spherical manner so that a wide range of rotation and orientation changes can be accommodated within the field of view of the imaging system. In still another embodiment, the passive or active point markers are affixed to the tool in a ring shape so that a wide range of rotation and orientation changes can be accommodated within the field of view of the imaging system.

Numerous additional useful features may be incorporated into the present invention. For example, for purposes of image filtering, band-pass or high-pass filters may be incorporated into the optical sequence for each of the plurality of digital cameras in data capturing component 200 for permitting light from only the wavelengths which are reflected or emitted from the point markers to improve image signal-to-noise ratio. Spurious data may be rejected by analyzing only image information obtained from within a dynamic region of interest having a limited offset from a previously known rigid-body locality. This dynamic region of interest may be incorporated into or otherwise predefined (i.e., preprogrammed as a box or region of width x and height y and centered on known positions of target 98) within the field of view of each digital camera such that image information is only processed from this predefined region. The region of interest will change as the rigid body moves and is therefore based on previously known locations of the rigid body. This approach allows the imaging system to view only pixels within the dynamic region of interest when searching for point markers while disregarding or blocking pixels in the larger image frame that are not included in the dynamic region of interest. Decreased processing time is a benefit of this aspect of the invention.

In some embodiments of the present invention, the position and orientation of the operation path, or a predetermined segment thereof, relative to the three-dimensional space viewable by the imaging system is obtained from a three-dimensional CAD model, the coordinate system of which is known relative to the coordinate system of the imaging system. The three-dimensional CAD model may also contain a definition of linear or curvilinear points which define the operation path segment and at least three calibration points are located on both the three-dimensional CAD model and on the fixture. A position and orientation shift may be applied to the three-dimensional CAD model by measuring the position of the at least three calibration points on the fixture with the imaging system and then comparing the measurements to the original calibration points of the three-dimensional CAD model. In other embodiments, the position and orientation of the linear or curvilinear operation path, or a predetermined segment thereof, relative to the three-dimensional space viewable by the imaging system may obtained using a three-dimensional CAD model, wherein the coordinate system of the three-dimensional CAD model relative to the coordinate system of the imaging system is predetermined, and wherein the weld locations on the three-dimensional CAD model are pre-defined. Regarding the CAD model creation, there is typically a one-to-one relationship between the CAD model and the part in question and a sequence of calibration may be an aspect of the welding exercise. The model exists is virtual space and the user instructs the system as to the location of the two points. A linkage is created to eliminate any variance between the CAD model and the part or particular datum on tooling is utilized. A procedure to teach the system offline may also be included.

One definition of an operation path for this invention describes a single continuous path for operation. In certain embodiments, the operation path is divided into separate segments for welds that traverse corners or change in general direction. In this context, points make up an operation path segment (at least two), and contiguous operation path segments make up an operation path chain. Thus, the position and orientation of the operation path may be made up of one or more operation path segments that form a chain, and consecutive segments share an operation path point at the end of one segment and the start of the next segment. In such embodiments, the system provides the ability to move between multiple calibration planes; each operation plane depends on which calibration plane is being utilized, and each operation path is tied to a predetermined coordinate system.

Furthermore, the exemplary weld characterization system 10 of FIG. 2 can be modified to better process the case of a round (e.g., circular) operation path, such as with pipe welding, and/or a more complex assembly of workpieces.

As shown in FIG. 13, the weld characterization system 10 includes a training stand similar or identical to the stand 20 shown in FIG. 2, having the substantially flat base 22. Furthermore, a frame 1302, cage, or the like is also interfaced with the stand 20 (either directly or indirectly). For example, the frame 1302 could be connected to the camera or imaging device support 26.

In some embodiments, the frame 1302 can be readily separated from the stand 20 to promote the portability of the system 10. The frame 1302 includes one or more legs 1304 for further supporting the frame 1302, along with the stand 20. The legs 1304 may be height-adjustable. The frame defines various locations as which cameras (e.g., digital high-speed-vision cameras) can be mounted. As shown in FIG. 13, a plurality of cameras 1306 are mounted on the frame 1302 as various locations (and elevations) around the workpiece 54. In this manner, the frame 1302 at least partially surrounds the workpiece 54. In some exemplary embodiments, at least half of the workpiece 54 (i.e., 180 degrees in the case of a pipe) is surrounded by the frame 1302. In some exemplary embodiments, at least 75% of the workpiece 54 (i.e., 270 degrees in the case of a pipe) is surrounded by the frame 1302. In some exemplary embodiments, substantially all (i.e., ˜100%) of the workpiece 54 is surrounded by the frame 1302.

The cameras 1306 form part of the data capturing component 200. In some exemplary embodiments, the weld characterization system 10 includes 2 or more (e.g., 2-20) cameras. In some exemplary embodiments, the weld characterization system 10 includes 4 or more cameras, 5 or more cameras, 6 or more cameras, 7 or more cameras, 8 or more cameras, 9 or more cameras, 10 or more cameras, 11 or more cameras, or 12 or more cameras. In some exemplary embodiments, the weld characterization system 10 includes at least 4 cameras, at least 5 cameras, at least 6 cameras, at least 7 cameras, at least 8 cameras, at least 9 cameras, at least 10 cameras, at least 11 cameras, or at least 12 cameras. After calibration of the cameras (as needed), the weld coupon 54 and the welding tool 10 are calibrated as described herein. The distribution of the cameras 1306 around the workpiece 54 (e.g., a pipe) allow for the accurate tracking of welding operations on the workpiece 54.

While the present invention has been illustrated by the description of exemplary embodiments thereof, and while the embodiments have been described in certain detail, it is not the intention of the Applicant to restrict or in any way limit the scope of the appended claims to such detail. Additional advantages and modifications will readily appear to those skilled in the art. Therefore, the invention in its broader aspects is not limited to any of the specific details, representative devices and methods, and/or illustrative examples shown and described. Accordingly, departures may be made from such details without departing from the spirit or scope of the applicant's general inventive concept.

Claims

1. A system for characterizing welding operations, comprising:

(a) a data generating component, wherein the data generating component further includes: (i) a fixture, wherein the geometric characteristics of the fixture are predetermined; (ii) a workpiece adapted to be mounted on the fixture, wherein the workpiece includes at least one joint to be welded, wherein the vector extending along the joint to be welded defines an operation path, and wherein the operation path is linear, curvilinear, circular, or a combination thereof; (iii) at least one calibration device, wherein each calibration device further includes at least two point markers integral therewith, and wherein the geometric relationship between the point markers and the operation path is predetermined; and (iv) a welding tool, wherein the welding tool is operative to form a weld at the joint to be welded, wherein the welding tool defines a tool point and a tool vector, and wherein the welding tool further includes a target attached to the welding tool, wherein the target further includes a plurality of point markers mounted thereon in a predetermined pattern, and wherein the predetermined pattern of point markers is operative to define a rigid body; and

(b) a data capturing component, wherein the data capturing component further includes an imaging system for capturing images of the point markers; and

(c) a data processing component, wherein the data processing component is operative to receive information from the data capturing component and then calculate: (i) the position and orientation of the operation path relative to the three-dimensional space viewable by the imaging system; (ii) the position of the tool point and orientation of the tool vector relative to the rigid body; and (iii) the position of the tool point and orientation of the tool vector relative to the operation path.

2. The system of claim 1, wherein the imaging system further includes a plurality of digital cameras.

3. The system of claim 2, wherein at least one filter is incorporated into the optical sequence for each of the plurality of digital cameras for permitting light from only the wavelengths which are reflected or emitted from the point markers for improving image signal-to-noise ratio.

4. The system of claim 2, wherein the imaging system further includes at least one dynamic region of interest viewable by the plurality of digital cameras, wherein the dynamic region of interest is determined by use of previously known positions for the rigid body, and wherein image information is gathered and processed only from within the dynamic region of interest.

5. The system of claim 2, wherein the plurality of digital cameras includes at least one digital camera positioned above the workpiece and at least one digital camera positioned below the workpiece.

6. The system of claim 1, wherein the workpiece comprises a pipe.

7. The system of claim 1, wherein the position and orientation of the operation path is calibrated using at least two point markers integral to a calibration device which is placed at a known translational and rotational offset to the fixture, and wherein the fixture holds the workpiece at a known translational and rotational offset to the operation path.

8. The system of claim 1, wherein the position and orientation of the operation path is calibrated using at least two point markers located on a fixture that holds the workpiece at a known translational and rotational offset to the operation path.

9. The system of claim 1, wherein the operation path is non-linear, wherein the position and orientation of the operation path in three-dimensional space may be mapped using a calibration device that includes at least two point markers, and wherein the operation path dictates the placement of the calibration device at multiple, different points thereon.

10. The system of claim 9, wherein the operation path is circular.

11. The system of claim 1, wherein the position and orientation of the operation path undergoes a predetermined translational and rotational offset from its original calibration plane based on predetermined sequence steps included in the overall system operation.

12. The system of claim 1, wherein the position and orientation of the operation path, or a predetermined segment thereof, relative to the three-dimensional space viewable by the imaging system is obtained from a three-dimensional CAD model, the coordinate system of which is known relative to the coordinate system of the imaging system.

13. The system of claim 12, wherein the three-dimensional CAD model contains a definition of linear or curvilinear points which define the operation path segment.

14. The system of claim 12, wherein at least three calibration points are located on both the three-dimensional CAD model and on the fixture.

15. The system of claim 14, wherein a position and orientation shift is applied to the three-dimensional CAD model by measuring the position of the at least three calibration points on the fixture with the imaging system and then comparing the measurements to the original calibration points of the three-dimensional CAD model.

16. The system of claim 1, wherein the position and orientation of the operation path is made up of one or more operation path segments that form a chain, and wherein consecutive segments share an operation path point at the end of one segment and the start of the next segment.

17. The system of claim 1, wherein calibration of the tool point and tool vector of the welding tool is performed using two or more point markers integrated into a removable calibration device, and wherein the point markers in the calibration device are located along a tool vector that has a known offset to the tool point of the welding tool.

18. The system of claim 1, wherein calibration of the tool point of the welding tool is performed by inserting the tip of the welding tool into a calibration device, the position and orientation of which relative to the workpiece is predetermined.

19. The system of claim 18, wherein the system calculates values for at least one of tool position, orientation, velocity, and acceleration with respect to the operation path, wherein these values are then compared to predetermined preferred values to determine deviations from known and preferred procedures, and wherein such deviations are used for at least one of assessing skill level, providing feedback for training, assessing progress toward a skill goal, and quality control purposes.

20. The system of claim 1, wherein the point markers defining the rigid body are affixed to the welding tool in a multi-faceted configuration that accommodates a wide range of rotation and orientation changes of the welding tool when in use.

21. The system of claim 1, wherein the point markers defining the rigid body are affixed to the welding tool in a spherical configuration that accommodates a wide range of rotation and orientation changes of the welding tool when in use.

22. The system of claim 1, wherein the passive or active point markers are affixed to the welding tool in a ring configuration that accommodates a wide range of rotation and orientation changes of the welding tool when in use.

23. The system of claim 1, wherein the system calculates values for at least one of tool position, orientation, velocity, and acceleration with respect to the operation path, wherein these values are then compared to predetermined preferred values to determine deviations from known and preferred procedures, and wherein such deviations are used for at least one of assessing skill level, providing feedback for training, assessing progress toward a skill goal, and quality control purposes.

24. The system of claim 1, wherein the fixture includes a frame;

wherein the frame at least partially surrounds the workpiece;

wherein the frame includes a plurality of digital cameras mounted thereon; and

wherein each digital camera captures images of the workpiece from a different angle.