Abstract: A method for providing video coding is disclosed. The method includes encoding at least one segment of a B picture using motion estimation based prediction. During the encoding step, a determination is made as to whether a number of motion vectors corresponding to a number of already constructed reference pictures of the B picture are acceptable for use in a direct bi-directional prediction mode without further refinement. The motion vectors are refined in response to the determination that the motion vectors are not acceptable. A mode is then selected to be used for coding the segment of the B picture. The mode is selected from a number of modes including a temporal direct mode and a number of standard modes including forward, backward and bi-directional predictive modes. The method further includes decoding the segment of the B picture in an encoded bit stream. During the decoding step, entropy decoding of the encoded bit stream is performed to recreate residual data.