Abstract: Embodiments of present disclosure relates to method and gesture prediction system for predicting gesture of subjects for controlling AV during navigation. The gesture prediction system obtains data from sensors and generates context information of environment of AV. The gesture prediction system generates parameters for sampling frames for determining gesture of subjects. The gesture prediction system estimates current pose, and subsequent poses by extrapolating current pose and subsequent poses using deep learning techniques. Further, the gesture prediction system infers current behaviour of subjects by classifying current pose and subsequent poses into one of predefined gesture categories. The gesture prediction system predicts gesture of subjects based on current behaviour, context information and location information of subjects for controlling AV during navigation. Thus, the present disclosure forecast gestures of subjects at a faster rate and controls the AV during the navigation.