Abstract: A method for learning parameters of CNNs capable of identifying objectnesses by detecting bottom lines and top lines of nearest obstacles in an input image is provided. The method includes steps of: a learning device, (a) instructing a first CNN to generate first encoded feature maps and first decoded feature maps, and instructing a second CNN to generate second encoded feature maps and second decoded feature maps; (b) generating first and second obstacle segmentation results respectively representing where the bottom lines and the top lines are estimated as being located per each column, by referring to the first and the second decoded feature maps respectively; (c) estimating the objectnesses by referring to the first and the second obstacle segmentation results; (d) generating losses by referring to the objectnesses and their corresponding GTs; and (f) backpropagating the losses, to thereby learn the parameters of the CNNs.

Abstract: A method for learning parameters of a CNN based on regression losses is provided. The method includes steps of: a learning device instructing a first to an n-th convolutional layers to generate a first to an n-th encoded feature maps; instructing an n-th to a first deconvolutional layers to generate an n-th to a first decoded feature maps from the n-th encoded feature map; generating an obstacle segmentation result by referring to a feature of the decoded feature maps; generating the regression losses by referring to differences of distances between each location of the specific rows, where bottom lines of nearest obstacles are estimated as being located per each of columns of a specific decoded feature map, and each location of exact rows, where the bottom lines are truly located per each of the columns on a GT; and backpropagating the regression losses, to thereby learn the parameters.

Abstract: A learning method for segmenting an image having one or more lanes is provided to be used for supporting collaboration with HD maps required to satisfy level 4 of autonomous vehicles. The learning method includes steps of: a learning device instructing a CNN module (a) to apply convolution operations to the image, thereby generating a feature map, and apply deconvolution operations thereto, thereby generating segmentation scores of each of pixels on the image; (b) to apply Softmax operations to the segmentation scores, thereby generating Softmax scores; and (c) to (I) apply multinomial logistic loss operations and pixel embedding operations to the Softmax scores, thereby generating Softmax losses and embedding losses, where the embedding losses is used to increase inter-lane differences among averages of the segmentation scores and decrease intra-lane variances among the segmentation scores, in learning parameters of the CNN module, and (II) backpropagate the Softmax and the embedding losses.

Abstract: A method of neural network operations by using a grid generator is provided for converting modes according to classes of areas to satisfy level 4 of autonomous vehicles. The method includes steps of: (a) a computing device instructing a pair detector to acquire information on locations and classes of pairs for testing by detecting the pairs for testing; (b) the computing device instructing the grid generator to generate section information by referring to the information on the locations of the pairs for testing; (c) the computing device instructing a neural network to determine parameters for testing by referring to parameters for training which have been learned by using information on pairs for training; and (d) the computing device instructing the neural network to apply the neural network operations to a test image by using each of the parameters for testing to thereby generate one or more neural network outputs.

Abstract: A learning method of a CNN capable of detecting one or more lanes is provided. The learning method includes steps of: a learning device (a) applying convolution operations to an image, to generate a feature map, and generating lane candidate information; (b) generating a first pixel data map including information on pixels in the image and their corresponding pieces of first data, wherein main subsets from the first data include distance values from the pixels to their nearest first lane candidates by Using a direct regression, and generating a second pixel data map including information on the pixels and their corresponding pieces of second data, wherein main subsets from the second data include distance values from the pixels to their nearest second lane candidates by using the direct regression; and (c) detecting the lanes by inference to the first pixel data map and the second pixel data map.

Abstract: A method for learning parameters of an object detector based on R-CNN is provided. The method includes steps of: a learning device (a) if training image is acquired, instructing (i) convolutional layers to generate feature maps by applying convolution operations to the training image, (ii) an RPN to output ROI regression information and matching information (iii) a proposal layer to output ROI candidates as ROI proposals by referring to the ROI regression information and the matching information, and (iv) a proposal-selecting layer to output the ROI proposals by referring to the training image; (b) instructing pooling layers to generate feature vectors by pooling regions in the feature map, and instructing FC layers to generate object regression information and object class information; and (c) instructing first loss layers to calculate and backpropagate object class loss and object regression loss, to thereby learn parameters of the FC layers and the convolutional layers.

Abstract: A method for learning an object detector based on an R-CNN by using a first to an n-th filter blocks respectively generating a first to an n-th feature maps through convolution operations in sequence, and a k-th to a first upsampling blocks respectively coupled with the first to the n-th filter blocks is provided. The method includes steps of: a learning device instructing the k-th upsampling block to the first upsampling block to generate a (k?1)-st pyramidic feature map to the first pyramidic feature map respectively; instructing an RPN to generate each ROI corresponding to each candidate region, and instructing a pooling layer to generate a feature vector; and learning parameters of the FC layer, the k-th to the first upsampling blocks, and the first to the n-th filter blocks by backpropagating a first loss generated by referring to object class information, object regression information, and their corresponding GTs.

Abstract: A method for warning a vehicle of a risk of lane change is provided. The method includes steps of: (a) an alarm device, if at least one rear image captured by a running vehicle is acquired, segmenting the rear image by using a learned convolutional neural network (CNN) to thereby obtain a segmentation image corresponding to the rear image; (b) the alarm device checking at least one free space ratio in at least one blind spot by referring to the segmentation image, wherein the free space ratio is determined as a ratio of a road area without an object in the blind spot to a whole area of the blind spot; and (c) the alarm device, if the free space ratio is less than or equal to at least one predetermined threshold value, warning a driver of the vehicle of the risk of lane change.

Abstract: A method for learning parameters of a CNN capable of detecting obstacles in a training image is provided. The method includes steps of: a learning device (a) receiving the training image and instructing convolutional layers to generate encoded feature maps from the training image; (b) instructing the deconvolutional layers to generate decoded feature maps; (c) supposing that each cell of a grid with rows and columns is generated by dividing the decoded feature map with respect to a direction of the rows and the columns, concatenating features of the rows per column in a direction of a channel, to generate a reshaped feature map; (d) calculating losses referring to the reshaped feature map and its GT image in which each row is indicated as corresponding to GT positions where a nearest obstacle is on column from its corresponding lowest cell thereof along the columns; and (e) backpropagating the loss.

Abstract: A method for encoding an image based on convolutional neural network is provided. The method includes steps of: a learning device a learning device including a first to an n-th convolutional layers, (a) obtaining at least one input image; (b) instructing each of at least one of the convolutional layers to (i) apply one or more transposed convolution operations to the input image or an input feature map received from its corresponding previous convolutional layer, to thereby generate one or more transposed feature maps which have different sizes respectively, and (ii) apply one or more convolution operations, with a different stride and a different kernel size, to their corresponding transposed feature maps, to thereby generate their corresponding one or more inception feature maps as a first group; and (c) concatenating or element-wise adding the inception feature maps included in the first group to thereby generate its corresponding output feature map.

Abstract: A learning method of a CNN capable of detecting one or more lanes using a lane model is provided. The method includes steps of: a learning device (a) acquiring information on the lanes from at least one image data set, wherein the information on the lanes are represented by respective sets of coordinates of pixels on the lanes; (b) calculating one or more function parameters of a lane modeling function of each of the lanes by using the coordinates of the pixels on the lanes; and (c) performing processes of classifying the function parameters into K cluster groups by using a clustering algorithm, assigning each of one or more cluster IDs to each of the cluster groups, and generating a cluster ID GT vector representing GT information on probabilities of being the cluster IDs corresponding to types of the lanes.

Abstract: A method for tracking an object by using a CNN including a tracking network is provided. The method includes steps of: a testing device (a) generating a feature map by using a current video frame, and instructing an RPN to generate information on proposal boxes; (b) (i) generating an estimated state vector by using a Kalman filter algorithm, generating an estimated bounding box, and determining a specific proposal box as a seed box, and (ii) instructing an FCN to apply full convolution operations to the feature map, to thereby output a position sensitive score map; (c) generating a current bounding box by referring to a regression delta and a seed box which are generated by instructing a pooling layer to pool a region, corresponding to the seed box, on the position sensitive score map, and adjusting the current bounding box by using the Kalman filter algorithm.

Abstract: A learning method of a CNN for detecting lanes is provided. The method includes steps of: a learning device (a) instructing convolutional layers to generate feature maps by applying convolution operations to an input image from an image data set; (b) instructing an FC layer to generate an estimated result vector of cluster ID classifications of the lanes by feeding a specific feature map among the feature maps into the FC layer; and (c) instructing a loss layer to generate a classification loss by referring to the estimated result vector and a cluster ID GT vector, and backpropagate the classification loss, to optimize device parameters of the CNN; wherein the cluster ID GT vector is GT information on probabilities of being cluster IDs per each of cluster groups assigned to function parameters of a lane modeling function by clustering the function parameters based on information on the lanes.

Abstract: A method for acquiring a pseudo-3D box from a 2D bounding box in a training image is provided. The method includes steps of: (a) a computing device acquiring the training image including an object bounded by the 2D bounding box; (b) the computing device performing (i) a process of classifying a pseudo-3D orientation of the object, by referring to information on probabilities corresponding to respective patterns of pseudo-3D orientation and (ii) a process of acquiring 2D coordinates of vertices of the pseudo-3D box by using regression analysis; and (c) the computing device adjusting parameters thereof by backpropagating loss information determined by referring to at least one of (i) differences between the acquired 2D coordinates of the vertices of the pseudo-3D box and 2D coordinates of ground truth corresponding to the pseudo-3D box, and (ii) differences between the classified pseudo-3D orientation and ground truth corresponding to the pseudo-3D orientation.

Abstract: A learning method for detecting a specific object based on convolutional neural network (CNN) is provided. The learning method includes steps of: (a) a learning device, if an input image is obtained, performing (i) a process of applying one or more convolution operations to the input image to thereby obtain at least one specific feature map and (ii) a process of obtaining an edge image by extracting at least one edge part from the input image, and obtaining at least one guide map including information on at least one specific edge part having a specific shape similar to that of the specific object from the obtained edge image; and (b) the learning device reflecting the guide map on the specific feature map to thereby obtain a segmentation result for detecting the specific object in the input image.

Abstract: A learning method for detecting at least one lane based on a convolutional neural network (CNN) is provided. The learning method includes steps of: (a) a learning device obtaining encoded feature maps, and information on lane candidate pixels in a input image; (b) the learning device, classifying a first parts of the lane candidate pixels, whose probability scores are not smaller than a predetermined threshold, as strong line pixels, and classifying the second parts of the lane candidate pixels, whose probability scores are less than the threshold but not less than another predetermined threshold, as weak lines pixels; and (c) the learning device, if distances between the weak line pixels and the strong line pixels are less than a predetermined distance, classifying the weak line pixels as pixels of additional strong lines, and determining that the pixels of the strong line and the additional correspond to pixels of the lane.

Abstract: A learning method for adjusting parameters of a CNN using loss augmentation is provided. The method includes steps of: a learning device acquiring (a) a feature map from a training image; (b) (i) proposal ROIs corresponding to an object using an RPN, and a first pooled feature map by pooling areas, on the feature map, corresponding to the proposal ROIs, and (ii) a GT ROI, on the training image, corresponding to the object, and a second pooled feature map by pooling an area, on the feature map, corresponding to the GT ROI; and (c) (i) information on pixel data of a first bounding box when the first and second pooled feature maps are inputted into an FC layer, (ii) comparative data between the information on the pixel data of the first bounding box and a GT bounding box, and backpropagating information on the comparative data to adjust the parameters.

Abstract: A learning method for improving image segmentation including steps of: (a) acquiring a (1-1)-th to a (1-K)-th feature maps through an encoding layer if a training image is obtained; (b) acquiring a (3-1)-th to a (3-H)-th feature maps by respectively inputting each output of the H encoding filters to a (3-1)-th to a (3-H)-th filters; (c) performing a process of sequentially acquiring a (2-K)-th to a (2-1)-th feature maps either by (i) allowing the respective H decoding filters to respectively use both the (3-1)-th to the (3-H)-th feature maps and feature maps obtained from respective previous decoding filters of the respective H decoding filters or by (ii) allowing respective K-H decoding filters that are not associated with the (3-1)-th to the (3-H)-th filters to use feature maps gained from respective previous decoding filters of the respective K-H decoding filters; and (d) adjusting parameters of CNN.

Abstract: A method for segmenting an image using a CNN including steps of: a segmentation device acquiring (i) a first segmented image for a t-th frame by a CNN_PREVIOUS, having at least one first weight learned at a t?(i+1)-th frame, segmenting the image, (ii) optical flow images corresponding to the (t?1)-th to the (t?i)-th frames, including information on optical flows from pixels of the first segmented image to corresponding pixels of segmented images of the (t?1)-th to the (t?i)-th frames, and (iii) warped images for the t-th frame by replacing pixels in the first segmented image with pixels in the segmented images referring to the optical flow images, (iv) losses by comparing the first segmented image with the warped images, (v) a CNN_CURRENT with at least one second weight obtained by adjusting the first weight to segment an image of the t-th frame and frames thereafter by using the CNN_CURRENT.

Abstract: A method for configuring a CNN with learned parameters that performs activation operation of an activation module and convolution operation of one or more convolutional layer in a convolutional layer at the same time is provided. The method includes steps of: (a) allowing a comparator to compare an input value corresponding to each of pixel values of an input image as a test image with a predetermined reference value and then output a comparison result; (b) allowing a selector to output a specific parameter corresponding to the comparison result among multiple parameters of the convolutional layer; and (c) allowing a multiplier to output a multiplication value calculated by multiplying the specific parameter by the input value and allowing the multiplication value to be determined as a result value acquired by applying the convolutional layer to an output of the activation module.