DATA STRUCTURE FOR CREATING IMAGE-PROCESSING DATA AND METHOD FOR CREATING IMAGE-PROCESSING DATA
Data structure for image-processing data is for creating image-processing data necessary for performing image processing on captured images of multiple workpieces when an articulated robot extracts a workable target workpiece from among multiple supplied workpieces. The data structure includes workpiece shape data for recognizing a target workpiece by pattern matching and tool data configured to check whether there is interference between a tool mounted on the articulated robot and a peripheral workpiece. The data structure is configured such that the combination of the workpiece shape data and the tool data can be rearranged for each workpiece or for each tool.
Latest FUJI CORPORATION Patents:
The present description discloses a data structure for creating image-processing data and a method for creating image-processing data.
BACKGROUND ARTConventionally, an industrial articulated robot capable of working on a workpiece has been known (for example, see Patent Literature 1).
Patent LiteraturePatent Literature 1: JP-A-2004-362018
SUMMARY OF THE INVENTION Technical ProblemSince the production of a large number of product types is often mixed in a modern multiproduct variable cell production line, it is desirable to have a large number of components (workpieces) of various product types supplied at the same time without changeover, and the method for arranging and supplying the workpieces is a major concern. One of the basic operations of an articulated robot is the operation of picking up and conveying a target workpiece. In the industrial robot in the related art, in order to facilitate picking up a workpiece with the articulated robot, the workpieces are often supplied in a state in which the direction and position of each workpiece are aligned by using a dedicated supply device. However, a dedicated supply device is not suitable for multiproduct variable cell production lines because it requires a dedicated design for each workpiece, and the space and costs of installation are not small.
For this reason, a simpler form of supplying workpieces is desired in which the workpieces are simply placed separately on a general-purpose supply device such as a case (component box) or a small conveyor so that a larger number of workpieces can be supplied with a small installation area without using a dedicated supply device. Dedicated supply devices supply workpieces at predetermined positions and orientations each time. Therefore, the articulated robot need only perform a predetermined pickup operation. However, in a general-purpose supply device, there are cases in which workpieces are loosely placed, the positions and orientations at which the workpieces are arranged are uncertain, and picking up a workpiece is impossible due to interference with workpieces that are not the pickup target. Therefore, it is necessary for the articulated robot to recognize the arrangement state of the workpiece and control the arm so as to obtain an appropriate pickup point and pickup orientation.
The arrangement state of the workpiece is automatically recognized by camera imaging and image processing in an image-processing device attached to the robot control device. The image-processing device recognizes the arrangement condition of the workpiece, that is, the position, direction, and orientation of the target workpiece and the interference state with non-target workpieces, and determines whether the target workpiece can be picked up. Generation of image-processing data for realizing such image processing usually requires a large amount of labor. However, in a production line for multiproduct variable production, efficient production of more product types by switching product types within a short time is required, and it is desirable to easily generate image-processing data of more product types in a short time.
A main object of the present disclosure is to provide a data structure or a creation method capable of creating image-processing data for various product types with a small amount of labor.
Solution to ProblemThe present disclosure employs the following means in order to achieve the above-mentioned main object.
The data structure for image-processing data creation of the present disclosure is a data structure for creating image-processing data necessary for performing image processing on captured images of multiple workpieces when an articulated robot extracts a workable target workpiece from among multiple supplied workpieces. The data structure includes workpiece shape data for recognizing the target workpiece by pattern matching and tool data for checking whether there is interference between a tool and a peripheral workpiece, and a combination of the workpiece shape data and the tool data can be rearranged for each workpiece or for each tool.
As a result, the data creator only needs to create the setting data once for the shared item, and thereafter, the setting data can be developed into various product types using the shared data. As a result, image-processing data for various product types can be created with less labor.
Next, embodiments of the present disclosure will be described with reference to the drawings.
Robot system 10 includes robot 20, robot control device 70, and image-processing device 80. In the present embodiment, robot system 10 is configured as a pick-and-place system that picks up workpiece W supplied by work supply device 12, and arranges and places picked up workpiece W on tray T conveyed by tray conveyance device 14. The robot system is not limited to the pick-and-place system and can be applied to any work system as long as work is performed on workpiece W using robot 20.
Robot 20 includes five-axis vertical articulated arm (hereinafter, referred to as arm) 22. Arm 22 has six links (first to sixth links 31 to 36) and five joints (first to fifth joints 41 to 45) for rotatably or pivotally connecting each link. Each of the joints (first to fifth joints 41 to 45) is provided with motors (servomotors) 51 to 55 for driving the corresponding joint and encoders (rotary encoders) 61 to 65 for detecting the rotational position of the corresponding motor.
Multiple types of pickup tools T1 to T3 as end effectors are detachably attached to the tip link (sixth link 36) of arm 22. In the present embodiment, pickup tool T1 is an electromagnetic chuck that picks up workpiece W made of a magnetic material with an electromagnet. Pickup tool T2 is a mechanical chuck having a pair of clamp claws movable between a nearby position for holding workpiece W and a separation position for releasing workpiece W. Pickup tool T3 is a suction nozzle that picks up workpiece W by negative pressure. The pickup tool to be mounted on the tip link is appropriately selected in accordance with the shape and material of workpiece W to be picked up.
Camera 24 is attached to a tip portion (fifth link 35) of arm 22. Camera 24 captures an image of each workpiece W in order to recognize the position and orientation of each workpiece W supplied by work supply device 12, and captures an image of tray T in order to recognize the position of tray T conveyed by tray conveyance device 14.
The proximal link (first link 31) of arm 22 is fixed to work table 11. Work supply device 12, tray conveyance device 14, and the like are disposed on work table 11. In the present embodiment, work supply device 12 is configured by a belt conveyor device including conveyor belt 13 laid across a driving roller and a driven roller, which are arranged apart from each other in the front-rear direction (Y-axis direction). Multiple workpieces W are separately placed on conveyor belt 13, and work supply device 12 supplies multiple workpieces W on conveyor belt 13 from the rear to the front by rotating and driving the driving roller. The work supply device may be a case supply device for supplying multiple workpieces accommodated in cases (component boxes), on a per case basis, instead of the belt conveyor device or in addition to the belt conveyor device. Tray conveyance device 14 is configured by a belt conveyor device and conveys tray T in a direction orthogonal to the supply direction of workpiece W (X-axis direction), and positions and holds tray T at a substantially central position.
Robot control device 70 is configured as a microprocessor centered on CPU 71 and includes ROM 72, HDD 73, RAM 74, an input and output interface (not shown), a communication interface (not shown), and the like in addition to CPU 71. Detection signals from encoders 61 to 65 and the like are input to robot control device 70. Robot control device 70 outputs control signals to work supply device 12, tray conveyance device 14, motors 51 to 55, tool actuator 56, and the like. Tool actuator 56 is an actuator for driving a pickup tool mounted on robot 20.
Robot control device 70 performs pickup process for causing robot 20 to pick up workpiece W and placing process for placing picked up workpiece W on tray T by driving and controlling motors 51 to 55 of robot 20. Specifically, the pickup process is performed as follows. That is, robot control device 70 acquires the target position of each joint of arm 22 corresponding to the target pickup position and orientation. Subsequently, robot control device 70 drives and controls the corresponding motors 51 to 55 so that the positions of the respective joints coincide with the acquired target positions. Robot control device 70 controls tool actuator 56 so that workpiece W is picked up by the pickup tool. Specifically, the placing process is performed as follows. That is, robot control device 70 acquires the target position of each joint of arm 22 corresponding to the target place position and orientation. Subsequently, robot control device 70 drives and controls the corresponding motors 51 to 55 so that the positions of the respective joints coincide with the acquired target positions. Then, robot control device 70 controls tool actuator 56 so that picked up workpiece W is placed (i.e., pickup of workpiece W is released).
Image-processing device 80 is configured as a microprocessor centered on CPU 81 and includes ROM 82, HDD 83, RAM 84, an input and output interface (not shown), a communication interface (not shown), and the like in addition to CPU 81. An image signal from camera 24, an input signal from input device 85, and the like are input to image-processing device 80. Image-processing device 80 outputs a drive signal to camera 24, an output signal to output device 86, and the like. Input device 85 is, for example, an input device such as a keyboard or a mouse, on which an operator performs an input operation. Output device 86 is a display device for displaying various information, such as a liquid crystal display, for example. Image-processing device 80 and robot control device 70 are communicably connected and exchange control signals and data with each other.
Image-processing device 80 transmits a control signal to robot control device 70 to move arm 22 (camera 24) to the imaging point of workpiece W supplied by work supply device 12, drives camera 24 to image workpiece W, and inputs the obtained image signal (captured image). Subsequently, image-processing device 80 processes the input image signal to recognize workpiece W in the captured image. Then, image-processing device 80 extracts a target workpiece, which can be picked up, from among recognized workpieces W, determines a target position and a target orientation of a pickup tool for picking up the target workpiece, and transmits the target position and the target orientation to robot control device 70. Such processing is performed in accordance with an image-processing sequence based on image-processing data.
The workpiece shape model is a template for pattern matching when recognizing workpiece W from the captured image. The workpiece shape model includes the contour shape of workpiece W and pickup position PP of workpiece W.
The pickup tool model includes interference check region AI for checking whether the pickup tool interferes with a peripheral workpiece when picking up a target workpiece with the pickup tool. Interference check region AI can be created by the shape of the tip end of the pickup tool and the shape of the range of motion (range of influence) of the pickup tool. Since the pickup tool model is different for each pickup tool type, a pickup tool model is created for each pickup tool type.
The image-processing setting model includes an image-processing sequence and parameter settings associated with the image sequence.
Next, image-processing device 80 sets search region AS in the captured image (S110). Search region AS is one of parameters determined by parameter setting, and defines a search range of workpiece W in the captured image. For example, in work supply device 12A shown in
Then, image-processing device 80 sets a workpiece/background condition for separating workpiece W and the background in the captured image (S130), and performs detection (S140) of the background color and detection (S150) of the workpiece color in the search region in accordance with the set workpiece/background condition. The workpiece/background condition is one of parameters determined by parameter setting, and includes a background color parameter designating a background color and a workpiece color parameter designating a workpiece color. The background color parameter is set for each work supply device to be used because the background of workpiece W may be different for each work supply device. The workpiece color parameter is also set for each workpiece W to be used because the color may be different for each workpiece W.
Next, image-processing device 80 performs contour extraction processing for extracting the contour (edges) of workpiece W in accordance with the contour extraction condition (S150). The contour extraction condition is one of parameters determined by parameter setting and includes a threshold used for contour extraction (contour extraction level threshold). Subsequently, image-processing device 80 determines whether the contour shape of the extracted workpiece W substantially matches the workpiece shape model (S160), determines whether another workpiece W is intruding into interference check region AI of extracted workpiece W (S170), and determines whether extracted workpiece W can be picked up based on the determination results (S180). Image-processing device 80 sets the target position and the target orientation of the pickup tool based on pickup position PP of workpiece W determined to be capable of being picked up, and performs coordinate transformation on the set target position and the set target orientation to the target position of motors 51 to 55 (S190). Then, image-processing device 80 transmits the coordinate-transformed target position to robot control device 70 (S200) and ends the image-processing sequence.
In the data structure of the present embodiment described above, the image-processing data is separated into a workpiece shape model, a pickup tool model, and an image-processing setting model, and is composed of a combination thereof. The image-processing data is configured so that models corresponding to each type of workpieces, each type of pickup tool, and each environmental condition of robot system 10 can be freely rearranged. If the data creator creates the image-processing data once, even if any of the types of workpieces, the type of pickup tool, and the environmental condition is changed thereafter, it is only necessary to newly create the item related to the change, and it is possible to easily develop the item to various product type variables. As a result, it is possible to create image-processing data of various product type variables with a small amount of labor.
In the data structure of the present embodiment, the image-processing data is separated into the workpiece shape model, the pickup tool model, and the image-processing setting model, and each model is configured to be recombinable, but only the workpiece shape model and the pickup tool model may be configured to be recombinable. The data structure in this case is preferably employed in a system in which the environmental conditions of robot system 10 do not change.
As described above, the data structure for image-processing data creation of the present disclosure is a data structure for creating image-processing data necessary for performing image processing on captured images of multiple workpieces when an articulated robot extracts a workable target workpiece from among multiple supplied workpieces. The data structure includes workpiece shape data for recognizing the target workpiece by pattern matching and tool data for checking whether there is interference between a tool and a peripheral workpiece, and a combination of the workpiece shape data and the tool data can be rearranged for each workpiece or for each tool.
In the data structure for image-processing data creation of the present disclosure, the workpiece shape data may include an outer shape of the workpiece and a tool work position at which the tool performs work on the workpiece.
Further, in the data structure for image-processing data creation of the present disclosure, the tool data may include an interference prohibition region (interference check region) in which interference of a peripheral workpiece with respect to the target workpiece is prohibited. Thus, even if the tool to be mounted is changed, the articulated robot can appropriately perform the work on the target workpiece without interfering with the peripheral workpiece. In this case, the interference prohibition region may be designated as a region obtained by setting a tool work position at which the tool performs work on the workpiece as a reference. By doing so, it is possible to appropriately link the workpiece shape data and the tool data. Further, in this case, the interference prohibition region may be designated as a circular region or a rectangular region. This makes it possible to create tool data with a smaller amount of data.
The data structure for image-processing data creation of the present disclosure may further include process data including a sequence of the image processing and parameter setting, in which a combination of the process data with the workpiece shape data and the tool data can be rearranged for each environmental condition. In this case, even when the environmental conditions are changed, since the existing data can be used as the workpiece shape data and the tool data by changing only the process data, the image-processing data can be created with a small amount of labor. The environmental condition may include a supply method of the workpiece or a lighting condition of the workpiece.
In the present disclosure, the present invention is not limited to the form of the data structure for image-processing data creation, but may be the form of the creation method of image-processing data.
The present invention is not limited in any way to the above-mentioned embodiments, and it is needless to say that the present invention can be implemented in various forms as long as it belongs to the technical scope of the invention of the present disclosure.
INDUSTRIAL APPLICABILITYThe present disclosure can be applied to an image-processing device, a manufacturing industry of an articulated robot, and the like.
REFERENCE SIGNS LIST10 robot system, 11 work table, 12, 12A, 12B work supply device, 13 conveyor belt, 14 tray conveyance device, 20 robot, 22 arm, 24 camera, 31 first link, 32 second link, 33 third link, 34 fourth link, 35 fifth link, 36 sixth link, 41 first joint, 42 second joint, 43 third joint, 44 fourth joint, 45 fifth joint, 51 to 55 motor, 56 tool actuator, 61 to 65 encoder, 70 robot control device, 71 CPU, 72 ROM, 73 HDD, 74 RAM, 80 image-processing device, 81 CPU, 82 ROM, 83 HDD, 84 RAM, 85 input device, 86 output device, W workpiece, T tray, T1 to T3 pickup tool, PP pickup position, AI interference check region, AS search region.
Claims
1.-8. (canceled)
9. A data structure configured to create image-processing data necessary for performing image processing on captured images of multiple workpieces when an articulated robot extracts a workable target workpiece from among multiple supplied workpieces, the data structure comprising:
- workpiece shape data for recognizing the target workpiece by pattern matching; and
- tool data for checking whether there is interference between a tool mounted on the articulated robot and a peripheral workpiece;
- wherein a combination of the workpiece shape data and the tool data can be rearranged for each workpiece or for each tool.
10. The data structure for image-processing data creation according to claim 9,
- wherein the workpiece shape data includes an outer shape of the workpiece and a tool work position at which the tool performs work on the workpiece.
11. The data structure for image-processing data creation according to claim 9,
- wherein the tool data includes an interference prohibition region in which interference of a peripheral workpiece with respect to the target workpiece is prohibited.
12. The data structure for image-processing data creation according to claim 11,
- wherein the interference prohibition region is designated as a region obtained by setting a tool work position in which the tool performs work on the workpiece as a reference.
13. The data structure for image-processing data creation according to claim 12,
- wherein the interference prohibition region is designated as a circular region or a rectangular region.
14. The data structure for image-processing data creation according to claim 9, further comprising:
- process data including a sequence of the image processing and parameter setting,
- wherein a combination of the process data with respect to the workpiece shape data and the tool data can be rearranged for each environmental condition.
15. The data structure for image-processing data creation according to claim 14,
- wherein the environmental condition includes a supply method of the workpiece or a lighting condition of the workpiece.
16. An image-processing data creation method for creating image-processing data necessary for performing image processing on captured images of multiple workpieces when an articulated robot extracts a workable target workpiece from among multiple supplied workpieces, the method comprising:
- creating workpiece shape data for recognizing the target workpiece by pattern matching and tool data for checking whether there is interference between a tool mounted on the articulated robot and a peripheral workpiece in advance and creating the image-processing data by combining the workpiece shape data and the tool data for each workpiece or for each tool.
Type: Application
Filed: Mar 6, 2017
Publication Date: Jan 23, 2020
Patent Grant number: 11222417
Applicant: FUJI CORPORATION (Chiryu-shi)
Inventor: Nobuo OISHI (Kosai-shi)
Application Number: 16/483,575