PROCESSING DEVICE OF DEPTH INFORMATION

Abstract

A depth information processing device includes a multi-directional information processor. The multi-directional information processor is used for generating a corrected depth value according to valid depth values of corresponding positions in at least two different directions corresponding to a position in a hole in depth information, wherein the corrected depth value is used for correcting an invalid depth value of the position, and the corresponding positions in the at least two different directions are outside the hole. After the multi-directional information processor corrects invalid depth values of a plurality of positions included in the hole, the multi-directional information processor outputs corrected depth information.

Description

CROSS REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional Application No. 63/184,792, filed on May 6, 2021. The content of the application is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a depth information processing device, and particularly to a depth information processing device that can fill a hole in depth information.

2. Description of the Prior Art

Nowadays, applications of some electronic products require depth information. However, when the electronic products generate the depth information, depth values of some positions in the depth information may be invalid (e.g. depth values of the positions cannot be measured or there is no depth value existing in the positions). That is, holes appear in areas corresponding to the positions in the depth information, resulting in the applications of the electronic products being invalid because of the holes.

The prior art utilizes a one-way search (e.g. from left to right) method to generate valid depth values corresponding to the positions to correct invalid depth values of the positions according to a plurality of first depth values corresponding to the positions in the depth information (wherein the plurality of first depth values are valid). However, the one-way search (e.g. from left to right) method may still produce invalid results in the depth information, resulting in the applications of the electronic products being still invalid. Therefore, how to design a depth information processing device that can fill the holes in the depth information has become an important issue.

SUMMARY OF THE INVENTION

An embodiment of the present invention provides a depth information processing device. The depth information processing device includes a multi-directional information processor. The multi-directional information processor is used for generating a corrected depth value according to valid depth values of corresponding positions in at least two different directions corresponding to a position in a hole in depth information, wherein the corrected depth value is used for correcting an invalid depth value of the position, and the corresponding positions in the at least two different directions are outside the hole. After the multi-directional information processor corrects invalid depth values of a plurality of positions included in the hole, the multi-directional information processor outputs corrected depth information.

According to one aspect of the present invention, the depth information processing device further includes an iteration processor, wherein the iteration processor is coupled to the multi-directional information processor, the iteration processor is used for controlling and making the multi-directional information processor restart to generate a first corrected depth value corresponding to a first position adjacent to the position at least according to the corrected depth value, the first position is included in the hole, and the first corrected depth value is used for correcting an invalid depth value of the first position.

According to one aspect of the present invention, the depth information processing device further includes a backtracking processor, wherein the backtracking processor is coupled to the multi-directional information processor, the backtracking processor is used for generating a first corrected depth value corresponding to a first position adjacent to the position at least according to the corrected depth value, and the first corrected depth value is used for substituting for a first invalid depth value of the first position.

Another embodiment of the present invention provides a depth information processing device. The depth information processing device includes an edge information processor. The edge information processor is used for utilizing valid depth values of corresponding positions in at least one direction corresponding to a position in a hole in depth information to generate a corrected depth value according to gray levels and/or color values of the corresponding positions in the at least one direction, wherein the corrected depth value is used for correcting an invalid depth value of the position, and the corresponding positions in the at least one direction are outside the hole. After the edge information processor corrects invalid depth values of a plurality of positions included in the hole, the edge information processor outputs corrected depth information.

According to one aspect of the present invention, the depth information processing device further includes an iteration processor, wherein the iteration processor is coupled to the edge information processor, the iteration processor is used for controlling and making the edge information processor restart to generate a first corrected depth value corresponding to a first position adjacent to the position at least according to the corrected depth value, the first position is included in the hole, and the first corrected depth value is used for correcting an invalid depth value of the first position.

According to one aspect of the present invention, the depth information processing device further includes a backtracking processor, wherein the backtracking processor is coupled to the edge information processor, the backtracking processor is used for generating a first corrected depth value corresponding to a first position adjacent to the position at least according to the corrected depth value, and the first corrected depth value is used for substituting for a first invalid depth value of the first position.

Another embodiment of the present invention provides a depth information processing device. The depth information processing device includes a filter. The filter is used for generating a filtering value corresponding to a filtering window in depth information according to valid depth values of a plurality of selected positions within the filtering window, wherein the filtering value is used for correcting an invalid depth value corresponding to a position within the filtering window in the depth information. After the filter corrects invalid depth values of a plurality of positions in the depth information, the filter outputs corrected depth information.

According to one aspect of the present invention, the depth information processing device further includes an iteration processor, wherein the iteration processor is coupled to the filter, the iteration processor is used for controlling and making the filter restart to generate a first filtering value corresponding to a first position of the plurality of positions adjacent to the position at least according to the filtering value, and the first filtering value is used for correcting an invalid depth value of the first position.

According to one aspect of the present invention, the depth information processing device further includes a backtracking processor, wherein the backtracking processor is coupled to the filter, the backtracking processor is used for generating a first corrected depth value corresponding to a first position adjacent to the position at least according to the filtering value, and the first filtering value is used for substituting for a first invalid depth value of the first position.

According to one aspect of the present invention, the filter is a medium filter, or a mean filter, or a Gaussian filter.

According to one aspect of the present invention, the filter first calculates weights of the plurality of selected positions according to at least one item of gray level values, color values, valid depth values, and position relationships of the plurality of selected positions, and then obtains the filtering value according to the weights of the plurality of selected positions and the valid depth values of the plurality of selected positions.

These and other objectives of the present invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment that is illustrated in the various figures and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a depth information processing device according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating two opposite directions corresponding to a position (with an invalid depth value) in a hole in the depth information.

FIG. 3 is a diagram illustrating the multi-directional information processor generating the corrected depth value according to valid depth values of the corresponding positions in the two opposite directions corresponding to the position in the hole in the depth information.

FIG. 4 is a diagram illustrating the corrected depth information.

FIG. 5 is a diagram illustrating the multi-directional information processor averaging valid depth values of corresponding positions at four different directions corresponding to the position to generate the corrected depth value.

FIG. 6 is a diagram illustrating a depth information processing device according to a second embodiment of the present invention.

FIG. 7 is a diagram illustrating operation of the iteration processor.

FIG. 8 is a diagram illustrating a depth information processing device according to a third embodiment of the present invention.

FIG. 9 is a diagram illustrating operation of the backtracking processor.

FIG. 10 is a diagram illustrating a depth information processing device according to a fourth embodiment of the present invention.

FIG. 11 is a diagram illustrating a relationship between gray level values of the positions in the depth information.

FIG. 12 is a diagram illustrating a relationship between gray level values of the object, the hole, and the background in the depth information.

FIG. 13 is a diagram illustrating the corrected depth information.

FIG. 14 is a diagram illustrating a depth information processing device according to a fifth embodiment of the present invention.

FIG. 15 is a diagram illustrating the positions within the filtering window.

DETAILED DESCRIPTION

Please refer to FIG. 1. FIG. 1 is a diagram illustrating a depth information processing device 100 according to a first embodiment of the present invention, wherein the depth information processing device 100 includes a multi-directional information processor 102. As shown in FIG. 2, the multi-directional information processor 102 can generate a corrected depth value to correct an invalid depth value of a position E according to valid depth values of corresponding positions at two opposite directions (e.g. a left direction LD and a right direction RD) corresponding to the position E (e.g. the position E can correspond to one (or more) pixel) in a hole in depth information DP (e.g. a depth map). For example, at the right direction RD, the multi-directional information processor 102 can generate a mean MR according to valid depth values of positions A, B, C, D, and at the left direction LD, can generate a mean ML according to valid depth values of positions A, B, C, F. Then, the multi-directional information processor 102 can generate the corrected depth value (e.g. finding a minimum value between the mean MR and the mean ML) according to the means MR, ML, wherein the multi-directional information processor 102 can utilize the corrected depth value to correct the invalid depth value of the position E. In addition, in another embodiment of the present invention, at the right direction RD, the multi-directional information processor 102 can obtain a minimum value MIR according to the valid depth values of the positions A, B, C, D, and at the left direction LD, the multi-directional information processor 102 can obtain a minimum value MIL according to the valid depth values of the positions A, B, C, F. Then, the multi-directional information processor 102 can generate the corrected depth value (e.g. finding a minimum value between the minimum value MIR and the minimum value MIL) according to the minimum value MIR and the minimum value MIL, wherein the multi-directional information processor 102 can utilize the corrected depth value to correct the invalid depth value of the position E. In addition, in the depth information, the positions A, B, C, D, E, F can correspond to one (or more) pixel. In addition, the present invention is not limited to the above-mentioned methods for generating the corrected depth value. For example, in another embodiment of the present invention, at the right direction RD, the multi-directional information processor 102 can generate a first median according to the valid depth values of the positions A, B, C, D, and at the left direction LD, generate a second median according to the valid depth values of the positions A, B, C, F. Then, the multi-directional information processor 102 can generate the corrected depth value according to the first median and the second median. In addition, in another embodiment of the present invention, the multi-directional information processor 102 can generate the corrected depth value simultaneously and directly according to the valid depth values of the positions A, B, C, D, F. In addition, the present invention is not limited to the multi-directional information processor 102 generating the corrected depth value only according to the valid depth values at the two opposite directions corresponding to the position E with the invalid depth value in the hole in the depth information DP. That is, in another embodiment of the present invention, the multi-directional information processor 102 can generate the corrected depth value according to valid depth values at multiple directions corresponding to the position E with the invalid depth value in the hole in the depth information DP.

In addition, in the present invention, the depth information DP can be any information with depth, for example, the depth information DP can be a depth map (parallax, or distance, or phase), or a point cloud (coordinates), or data generated by any depth device (e.g. binocular vision, or Time of Flight (ToF), or Lidar, or Radar, or sound waves). The positions A, B, C, D, E, Fare defined as sampling positions within the depth information DP. For example, sampling positions within the depth map are pixels, sampling positions within the point cloud are coordinates or voxels. In addition, the two opposite directions in the depth information DP are defined as specific vectors after dimension reduction corresponding to the position E.

In addition, in another embodiment of the present invention, as shown in FIG. 3 (wherein in FIG. 3, the depth information DP takes the depth map as an example), multi-directional information processor 102 can generate a corrected depth value according to valid depth values of corresponding positions at two opposite directions (e.g. a left direction LD and a right direction RD) corresponding to a position P (e.g. the position P can correspond to one (or more) pixel) in a hole HO in a depth map DP, wherein the corrected depth value is used for correcting an invalid depth value of the position P, and depth information DP includes an object 202. For example, as shown in FIG. 3, multi-directional information processor 102 can generate the corrected depth value according to the valid depth values of the corresponding positions (e.g. positions PL, PR) at the two opposite directions (e.g. the left direction LD and the right direction RD) corresponding to the position P, wherein the corrected depth value is used for correcting the invalid depth value of the position P, the positions PL, PR are outside the hole HO, and the position PL (the position PR) can correspond to one (or more) pixel. In one embodiment of the present invention, the multi-directional information processor 102 averages the valid depth values of the positions PL, PR to generate the corrected depth value, But, in another embodiment of the present invention, multi-directional information processor 102 can average valid depth values of at least three corresponding positions (e.g. positions PL, PR, PL1, PR1) at the two opposite directions (e.g. the left direction LD and the right direction RD) corresponding to the position P to generate the corrected depth value. But, in another embodiment of the present invention, the multi-directional information processor 102 utilizes a minimum value between the valid depth values of the positions PL, PR to generate the corrected depth value (that is, the minimum value between the valid depth values of the positions PL, PR is the corrected depth value). In addition, after the multi-directional information processor 102 corrects invalid depth values of a plurality of positions in the hole HO, the multi-directional information processor 102 outputs corrected depth information CDP (as shown in FIG. 4). In addition, the present invention is not limited to multi-directional information processor 102 only averaging the valid depth values of the corresponding positions (e.g. the positions PL, PR) at the two opposite directions (e.g. the left direction LD and the right direction RD) corresponding to the position P to generate the corrected depth value. That is, in another embodiment of the present invention, the multi-directional information processor 102 can average valid depth values of corresponding positions at more than two directions corresponding to the position P to generate the corrected depth value. For example, as shown in FIG. 5, the multi-directional information processor 102 can average valid depth values of corresponding positions at four different directions (an upper direction UD, a down direction DD, the left direction LD, and the right direction RD) corresponding to the position P to generate the corrected depth value.

In addition, because one of ordinary skilled in the art should know that the multi-directional information processor 102 can be a field programmable gate array (FPGA) with the above-mentioned functions of the multi-directional information processor 102, or an application-specific integrated circuit (ASIC) with the above-mentioned functions of the multi-directional information processor 102, or a software module with the above-mentioned functions of the multi-directional information processor 102, or an analog integrated circuit with the above-mentioned functions of the multi-directional information processor 102 according to the above-mentioned functions of the multi-directional information processor 102, descriptions of a corresponding structure of the multi-directional information processor 102 are omitted for simplicity.

Please refer to FIG. 6. FIG. 6 is a diagram illustrating a depth information processing device 600 according to a second embodiment of the present invention, wherein a difference between the depth information processing device 600 and the depth information processing device 100 is that the depth information processing device 600 further includes an iteration processor 602, wherein the iteration processor 602 is coupled to the multi-directional information processor 102. Please refer to FIG. 7. FIG. 7 is a diagram illustrating operation of the iteration processor 602. As shown in FIG. 7(a), there are 9 positions in depth information DP, wherein if the depth information DP is a depth map, each position of the 9 positions corresponds to one (or more) pixel, and if the depth information DP is a point cloud, each position of the 9 positions corresponds to one coordinate area (including one or more coordinates) or one voxel. In addition, a depth value in the depth map can be parallax or a distance, a depth value in the point cloud can be coordinate information or coordinate information plus color, normal vector, texture information, etc. In addition, when the depth information DP includes a hole, the hole in a depth map is usually represented by invalid depth values, and the hole in a point cloud indicates that there is no point cloud (no coordinate information, etc.) in the hole or voxel. In addition, in order to simplify the description, invalid depth values in the point cloud mentioned hereafter has no coordinate information. As shown in FIG. 7(a), positions A, B, C, D in the hole have invalid depth values, wherein the positions A, B, C, D are represented by slashes. First, the multi-directional information processor 102 can generate corrected depth values corresponding to the positions A, B, C to correct the invalid depth values of the positions A, B, C according to corresponding descriptions in FIGS. 2, 3, 4, 5 and valid depth values of positions E1, E2, E3, E4, E5 (wherein it is assumed that the multi-directional information processor 102 can only correct depth values of positions adjacent to the position E1, E2, E3, E4, E5). Meanwhile, as shown in FIG. 7(b), only the position D in the hole has invalid depth value, that is, only the position D is represented by slashes. After, the iteration processor 602 controls and makes the multi-directional information processor 102 restart to generate a corrected depth value (that is, a first corrected depth value) corresponding to the position D (that is, a first position) in the hole at least according to the value positions A, B, C, wherein the position D is adjacent to the positions A, B, C, and the corrected depth value corresponding to the position D is used for correcting an invalid depth value of the position D. Meanwhile, as shown in FIG. 7(c), no position in the hole has invalid depth value, that is, no position needs to be represented by slashes.

In addition, because one of ordinary skilled in the art should know that the iteration processor 602 can be a field programmable gate array (FPGA) with the above-mentioned functions of the iteration processor 602, or an application-specific integrated circuit (ASIC) with the above-mentioned functions of the iteration processor 602, or a software module with the above-mentioned functions of the iteration processor 602, or an analog integrated circuit with the above-mentioned functions of the iteration processor 602 according to the above-mentioned functions of the iteration processor 602, descriptions of a corresponding structure of the iteration processor 602 are omitted for simplicity.

Please refer to FIG. 8. FIG. 8 is a diagram illustrating a depth information processing device 800 according to a third embodiment of the present invention, wherein a difference between the depth information processing device 800 and the depth information processing device 100 is that the depth information processing device 800 further includes a backtracking processor 802, and the backtracking processor 802 is coupled to the multi-directional information processor 102. Please refer to FIG. 9. FIG. 9 is a diagram illustrating operation of the backtracking processor 802. As shown in FIG. 9, positions B, C in a hole in a depth information DP have invalid depth values, wherein the positions B, C are represented by slashes, and each position of positions A, B, C, D corresponds to one (or more) pixel. First, the multi-directional information processor 102 can generate a corrected depth value corresponding to the position B to substitute for an invalid depth value of the position B according to the corresponding descriptions in FIGS. 2, 3, 4, 5 and a valid depth value of the position A. That is, meanwhile, the corrected depth value of the position B has substituted for the invalid depth value of the position B, and the corrected depth value of the position B is valid. After, the backtracking processor 802 can generate a corrected depth value (i.e. a first corrected depth value) corresponding to the position C (i.e. a first position) to substitute for an invalid depth value of the position C according to the corresponding descriptions in FIGS. 2, 3, 4, 5 and the corrected depth value of the position B (or according to the corresponding descriptions in FIGS. 2, 3, 4, 5, the corrected depth value of the position B, and the valid depth value of the position A). Similarly, the multi-directional information processor 102 can generate the corrected depth value corresponding to the position C to substitute for the invalid depth value of the position C according to the corresponding descriptions in FIGS. 2, 3, 4, 5 and a valid depth value of the position D. That is, meanwhile, the corrected depth value of the position C has substituted for the invalid depth value of the position C, and the corrected depth value of the position C is valid. After, the backtracking processor 802 can generate the corrected depth value corresponding to the position B to substitute for the invalid depth value of the position B according to the corresponding descriptions in FIGS. 2, 3, 4, 5 and the corrected depth value of the position C (or according to the corresponding descriptions in FIGS. 2, 3, 4, 5, the corrected depth value of the position C, and the valid depth value of the position D).

In addition, because one of ordinary skilled in the art should know that the backtracking processor 802 can be a field programmable gate array (FPGA) with the above-mentioned functions of the backtracking processor 802, or an application-specific integrated circuit (ASIC) with the above-mentioned functions of the backtracking processor 802, or a software module with the above-mentioned functions of the backtracking processor 802, or an analog integrated circuit with the above-mentioned functions of the backtracking processor 802 according to the above-mentioned functions of the backtracking processor 802, descriptions of a corresponding structure of the backtracking processor 802 are omitted for simplicity.

In addition, in another embodiment of the present invention, a depth information processing device can include the multi-directional information processor 102, the iteration processor 602, and the backtracking processor 802, wherein the iteration processor 602 can be coupled between the backtracking processor 802 and the multi-directional information processor 102, or the backtracking processor 802 can be coupled between the iteration processor 602 and the multi-directional information processor 102.

Please refer to FIG. 10. FIG. 10 is a diagram illustrating a depth information processing device 1000 according to a fourth embodiment of the present invention, wherein the depth information processing device 1000 includes an edge information processor 1002. As shown in FIG. 11(a), a position E in a hole in a depth information DP has an invalid depth value, wherein the position E is represented by slashes, and each position of positions A, B, C, D, E, F corresponds to one or more information (e.g. a depth value, a gray level value, or a color value, etc.). In addition, as shown in FIG. 11(b), gray level values (or color values, or gray level values and color values) corresponding to the positions A, C, D, F are closer, gray level values (or color values, or gray level values and color values) the positions B, E are closer, and the gray level value (or color value, or gray level value and color value) corresponding to the position E is very different from the gray level values (or color values, or gray level values and color values) corresponding to the positions D, F. Therefore, the edge information processor 1002 can judge a determined direction through information (e.g. gray level values) corresponding to the positions A˜F, and can generate a corrected depth value according to a valid depth value position B (at the determined direction and above the position E) in the hole in the depth information DP, wherein the corrected depth value is used for correcting an invalid depth value of the position E. In addition, after the edge information processor 1002 corrects the invalid depth value of the position E, the edge information processor 1002 can output corrected depth information CDP. In addition, in another embodiment of the present invention, when the position B corresponds to multiple information (e.g. depth value, gray level value, or color value, etc.), the edge information processor 1002 can generate the corrected depth value according to the corresponding descriptions in FIGS. 2, 3, 4, 5 and multiple valid depth values corresponding to the position B.

In addition, as shown in FIG. 12(a), an object 1202 within a depth information DP has a hole HO, and as shown in FIG. 12(b), gray level values (or color values, or gray level values and color values) corresponding to positions within the object 1202 are closer each other, gray level values (or color values, or gray level values and color values) corresponding to positions within a background 1204 are closer each other, and the gray level values (or the color values, or the gray level values and the color values) corresponding to the positions within the object 1202 are very different from the gray level values (or the color values, or the gray level values and the color values) corresponding to the positions within the background 1204. Therefore, the edge information processor 1002 can generate a plurality of corrected depth values corresponding to the hole HO according to multiple valid depth values of multiple positions within the object 1202 above the hole HO and/or multiple valid depth values of multiple positions within the object 1202 below the hole HO, wherein the plurality of corrected depth values are used for correcting invalid depth values of a plurality of positions of the hole HO to generate corrected depth information CDP (as shown in FIG. 13).

In addition, because one of ordinary skilled in the art should know that the edge information processor 1002 can be a field programmable gate array (FPGA) with the above-mentioned functions of the edge information processor 1002, or an application-specific integrated circuit (ASIC) with the above-mentioned functions of the edge information processor 1002, or a software module with the above-mentioned functions of the edge information processor 1002, or an analog integrated circuit with the above-mentioned functions of the edge information processor 1002 according to the above-mentioned functions of the edge information processor 1002, descriptions of a corresponding structure of the edge information processor 1002 are omitted for simplicity.

In addition, in another embodiment of the present invention, a depth information processing device can include the edge information processor 1002, iteration processor 602, and the backtracking processor 802, wherein the iteration processor 602 can be coupled between the backtracking processor 802 and the edge information processor 1002, or the backtracking processor 802 can be coupled between the iteration processor 602 and the edge information processor 1002. In addition, the iteration processor 602 can be referred to corresponding descriptions in FIGS. 6, 7, so further descriptions are omitted for simplicity; and the backtracking processor 802 can be referred to corresponding descriptions in FIGS. 8, 9, so further descriptions are also omitted for simplicity.

Please refer to FIG. 14. FIG. 14 is a diagram illustrating a depth information processing device 1400 according to a fifth embodiment of the present invention, wherein the depth information processing device 1400 includes a filter 1402, and the filter 1402 can be a medium filter, or a mean filter, or a Gaussian filter, or another filter. As shown in FIG. 15, depth values of positions P, P8 in a filtering window 1502 (if the depth information DP is a depth map, the filtering window is usually two-dimensional, and if the depth information DP is a point cloud, the filtering window is usually three-dimensional) within a depth information DP are invalid, and depth values of a plurality of selected positions P1˜P7 in the filtering window 1502 within the depth information DP are valid. The filter 1402 can first substitute differences between gray level values (or color values, or gray level values and color values) corresponding to the plurality of selected positions P1˜P7 and a gray level value (or color value, or gray level value and color value) corresponding to the position P into a distribution curve of the filter 1402 to generate weights corresponding to the plurality of selected positions P1˜P7, wherein because the depth value of the position P8 is invalid, the filter 1402 neglects the position P8. After, the filter 1402 generates a filtering value corresponding to the filtering window 1502 according to the weights corresponding to the plurality of selected positions P1˜P7 and the depth values of a plurality of selected positions P1˜P7, wherein the filtering value is used for correcting the depth value of the position P. After the filter 1402 utilizes the filtering window 1502 to correct invalid depth values of a plurality of positions in the depth information DP, the filter 1402 outputs corrected depth information CDP.

In addition, because one of ordinary skilled in the art should know that the filter 1402 can be a field programmable gate array (FPGA) with the above-mentioned functions of the filter 1402, or an application-specific integrated circuit (ASIC) with the above-mentioned functions of the filter 1402, or a software module with the above-mentioned functions of the filter 1402, or an analog integrated circuit with the above-mentioned functions of the filter 1402 according to the above-mentioned functions of the filter 1402, descriptions of a corresponding structure of the filter 1402 are omitted for simplicity.

In addition, in another embodiment of the present invention, a depth information processing device can include the filter 1402, the iteration processor 602, and the backtracking processor 802, wherein the iteration processor 602 can be coupled between the backtracking processor 802 and the filter 1402, or the backtracking processor 802 can be coupled between the iteration processor 602 and the filter 1402. In addition, the iteration processor 602 can be referred to the corresponding descriptions in FIGS. 6, 7, so further descriptions are omitted for simplicity; and the backtracking processor 802 can be referred to the corresponding descriptions in FIGS. 8, 9, so further descriptions are also omitted for simplicity.

In addition, all of the above-mentioned embodiments can determine whether to correct the invalid depth values at the beginning or the end. That is to say, when the corrected depth value may be incorrect or the hole is not suitable for correction, the hole can be accepted.

To sum up, the depth information processing device provided by the present invention can correct invalid depth values of a plurality of positions in the hole according to multi-directional information, or edge information, or filtering information corresponding to the hole in the depth information. Therefore, compared to the prior art, the depth information processing device provided by the present invention can more effectively fill the invalid depth values of the hole in the depth information.

Although the present invention has been illustrated and described with reference to the embodiments, it is to be understood that the invention is not to be limited to the disclosed embodiments, but on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A depth information processing device, comprising:

a multi-directional information processor generating a corrected depth value according to valid depth values of corresponding positions in at least two different directions corresponding to a position in a hole in depth information, wherein the corrected depth value is used for correcting an invalid depth value of the position, and the corresponding positions in the at least two different directions are outside the hole;

wherein after the multi-directional information processor corrects invalid depth values of a plurality of positions comprised in the hole, the multi-directional information processor outputs corrected depth information.

2. The depth information processing device of claim 1, further comprising:

an iteration processor coupled to the multi-directional information processor, wherein the iteration processor is used for controlling and making the multi-directional information processor restart to generate a first corrected depth value corresponding to a first position adjacent to the position at least according to the corrected depth value, the first position is comprised in the hole, and the first corrected depth value is used for correcting an invalid depth value of the first position.

3. The depth information processing device of claim 1, further comprising:

a backtracking processor coupled to the multi-directional information processor, wherein the backtracking processor is used for generating a first corrected depth value corresponding to a first position adjacent to the position at least according to the corrected depth value, and the first corrected depth value is used for substituting for a first invalid depth value of the first position.

4. A depth information processing device, comprising:

an edge information processor utilizing valid depth values of corresponding positions in at least one direction corresponding to a position in a hole in depth information to generate a corrected depth value according to gray levels and/or color values of the corresponding positions in the at least one direction, wherein the corrected depth value is used for correcting an invalid depth value of the position, and the corresponding positions in the at least one direction are outside the hole;

wherein after the edge information processor corrects invalid depth values of a plurality of positions comprised in the hole, the edge information processor outputs corrected depth information.

5. The depth information processing device of claim 4, further comprising:

an iteration processor coupled to the edge information processor, wherein the iteration processor is used for controlling and making the edge information processor restart to generate a first corrected depth value corresponding to a first position adjacent to the position at least according to the corrected depth value, the first position is comprised in the hole, and the first corrected depth value is used for correcting an invalid depth value of the first position.

6. The depth information processing device of claim 4, further comprising:

a backtracking processor coupled to the edge information processor, wherein the backtracking processor is used for generating a first corrected depth value corresponding to a first position adjacent to the position at least according to the corrected depth value, and the first corrected depth value is used for substituting for a first invalid depth value of the first position.

7. A depth information processing device, comprising:

a filter generating a filtering value corresponding to a filtering window in depth information according to valid depth values of a plurality of selected positions within the filtering window, wherein the filtering value is used for correcting an invalid depth value corresponding to a position within the filtering window in the depth information;

wherein after the filter corrects invalid depth values of a plurality of positions in the depth information, the filter outputs corrected depth information.

8. The depth information processing device of claim 7, further comprising:

an iteration processor coupled to the filter, wherein the iteration processor is used for controlling and making the filter restart to generate a first filtering value corresponding to a first position of the plurality of positions adjacent to the position at least according to the filtering value, and the first filtering value is used for correcting an invalid depth value of the first position.

9. The depth information processing device of claim 7, further comprising:

a backtracking processor coupled to the filter, wherein the backtracking processor is used for generating a first corrected depth value corresponding to a first position adjacent to the position at least according to the filtering value, and the first filtering value is used for substituting for a first invalid depth value of the first position.

10. The depth information processing device of claim 7, wherein the filter is a medium filter, or a mean filter, or a Gaussian filter.

11. The depth information processing device of claim 7, wherein the filter first calculates weights of the plurality of selected positions according to at least one item of gray level values, color values, valid depth values, and position relationships of the plurality of selected positions, and then obtains the filtering value according to the weights of the plurality of selected positions and the valid depth values of the plurality of selected positions.