Based on the orbital angular momentum and pulse position modulation, we present a novel passive measure-ment-device-independent quantum key distribution (MDI-QKD) scheme with the two-mode source. Combining with the tight bounds of the yield and error rate of single-photon pairs given in our paper, we conduct performance analysis on the scheme with heralded single-photon source. The numerical simulations show that the performance of our scheme is significantly superior to the traditional MDI-QKD in the error rate, key generation rate and secure transmis-sion distance, since the application of orbital angular momentum and pulse position modulation can exclude the ba-sis-dependent flaw and increase the information content for each single photon. Moreover, the performance is im-proved with the rise of the frame length. Therefore, our scheme, without intensity modulation, avoids the source side channels and enhances the key generation rate. It has greatly utility value in the MDI-QKD setups.