Abstract: The present invention provides an image calibrating method of a panoramic fish-eye camera comprising the following steps of: establishing a panoramic optical target space; using the panoramic fish-eye camera for shooting the panoramic optical target space's panoramic image; establishing a depth calibration parameter by using a depth calibration module; establishing an internal calibration parameter of the panoramic fish-eye camera; establishing an image stitching parameter and a space depth transformation parameter of the panoramic image and the panoramic optical target space; and using the internal calibration parameter, the image stitching parameter, the space depth transformation parameter and the depth calibration parameter to calibrate the panoramic image for generating a 3D panoramic image. Compared to the prior art, the present invention can optimize the calibration parameters by accumulating all the camera parameters and executing a machine learning for increasing the computing efficiency.