Sorry for my very late response (I have been in the hospital and was not able to answer). I guess this is already fixed. Please check with the latest version of bidi package. The issue was actually because of a xetex bug in which directionality gets lost in nested \hbox and \vbox.